00000100:00100000:3.0F:1787096277.488534:0:3626:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7983923b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3626:1873906084456960:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1787096277.488619:0:3626:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0F:1787096277.488656:0:7407:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084456960 00000100:00100000:1.0:1787096277.488672:0:7407:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084456960 00000100:00100000:1.0:1787096277.488715:0:7407:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 137 00000100:00100000:1.0:1787096277.488726:0:7407:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983342a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+5:3626:x1873906084456960:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1787096277.488758:0:7407:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446223872 avail 1299984384 00000100:00100000:1.0:1787096277.488806:0:7407:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983342a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+5:3626:x1873906084456960:12345-0@lo:41:osp-pre-0-1.0 Request processed in 84us (188us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096277.488813:0:7407:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 137 00000100:00100000:3.0:1787096277.488851:0:3626:0:(client.c:3102:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967358 gen 2 00000004:00080000:3.0:1787096277.488857:0:3626:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294967358 committed transno 4294967358 00000100:00100000:3.0:1787096277.488914:0:3626:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7983923b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3626:1873906084456960:0@lo:41:osp-pre-0-1.0 00010000:00010000:0.0F:1787096277.681452:0:6466:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787096277.735172:0:3621:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:1.0:1787096277.735252:0:10275:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077057408 00000100:00100000:1.0:1787096277.735262:0:10275:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077057408 00000100:00100000:1.0:1787096277.735317:0:10275:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 155 00000100:00100000:1.0:1787096277.735326:0:10275:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7abd5c0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f9858421-28a9-4744-99eb-72e5bedc52c4+9:34143:x1873906077057408:12345-192.168.201.52@tcp:101:rm.0 00010000:00010000:1.0:1787096277.735346:0:10275:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787096277.735417:0:10275:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7983881800/0x9eebb2eb7f50d741 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x4ef7bc65b7b410d6 expref: -99 pid: 10275 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.735478:0:10275:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7983880000/0x9eebb2eb7f50d748 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.735490:0:10275:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff8c7983880000/0x9eebb2eb7f50d748 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10275 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.735499:0:10275:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7983880000/0x9eebb2eb7f50d748 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10275 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787096277.735587:0:10275:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x1:0x0]: returning size 4096 00000004:00200000:1.0:1787096277.735592:0:10275:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1787096277.735630:0:10275:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff8c7983880000/0x9eebb2eb7f50d748 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10275 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.735661:0:10275:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7983881800/0x9eebb2eb7f50d741 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410d6 expref: 11 pid: 10275 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.735686:0:10275:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7983880000/0x9eebb2eb7f50d748 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410d6 expref: 10 pid: 10275 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.735693:0:10275:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff8c7a8b27af00) 00010000:00010000:1.0:1787096277.735696:0:10275:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787096277.735699:0:10275:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7983880000, rc 0) 00000100:00100000:1.0:1787096277.735751:0:10275:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7abd5c0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f9858421-28a9-4744-99eb-72e5bedc52c4+10:34143:x1873906077057408:12345-192.168.201.52@tcp:101:rm.0 Request processed in 427us (585us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096277.735757:0:10275:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 155 00000100:00100000:3.0:1787096277.736835:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:1.0:1787096277.736887:0:6478:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077057536 00000100:00100000:1.0:1787096277.736894:0:6478:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077057536 00000100:00100000:1.0:1787096277.736939:0:6478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 156 00000100:00100000:1.0:1787096277.736948:0:6478:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7982f50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f9858421-28a9-4744-99eb-72e5bedc52c4+10:34143:x1873906077057536:12345-192.168.201.52@tcp:101:rm.0 00010000:00010000:1.0:1787096277.736960:0:6478:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787096277.736973:0:6478:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983880a00/0x9eebb2eb7f50d74f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x4ef7bc65b7b410dd expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.737002:0:6478:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983880400/0x9eebb2eb7f50d756 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.737009:0:6478:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983880400/0x9eebb2eb7f50d756 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.737015:0:6478:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983880400/0x9eebb2eb7f50d756 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787096277.737034:0:6478:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000401:0x2:0x0]: returning size 4096 00000004:00200000:1.0:1787096277.737043:0:6478:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787096277.737063:0:6478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983880a00/0x9eebb2eb7f50d74f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410dd expref: 12 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.737073:0:6478:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983880400/0x9eebb2eb7f50d756 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410dd expref: 11 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.737078:0:6478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2:0x0].0x0 (ffff8c7a8b27a100) 00010000:00010000:1.0:1787096277.737081:0:6478:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787096277.737082:0:6478:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7983880400, rc 0) 00000100:00100000:1.0:1787096277.737111:0:6478:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7982f50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f9858421-28a9-4744-99eb-72e5bedc52c4+11:34143:x1873906077057536:12345-192.168.201.52@tcp:101:rm.0 Request processed in 167us (279us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096277.737116:0:6478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 156 00000100:00100000:2.0F:1787096277.738608:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:0.0:1787096277.738669:0:6477:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077057792 00000100:00100000:0.0:1787096277.738675:0:6477:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077057792 00000100:00100000:0.0:1787096277.738713:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 157 00000100:00100000:0.0:1787096277.738724:0:6477:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983920700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f9858421-28a9-4744-99eb-72e5bedc52c4+10:34143:x1873906077057792:12345-192.168.201.52@tcp:101:rm.0 00010000:00010000:0.0:1787096277.738733:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096277.738744:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd763400/0x9eebb2eb7f50d75d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x4ef7bc65b7b410e4 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787096277.738816:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x1:0x0]: returning size 4096 00010000:00010000:0.0:1787096277.738894:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd763400/0x9eebb2eb7f50d75d lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410e4 expref: 12 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.738904:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8c7a92c71d00) 00010000:00010000:0.0:1787096277.738906:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787096277.738909:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd763400/0x9eebb2eb7f50d75d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410e4 expref: 12 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.738919:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7abd763400, rc 0) 00000100:00100000:0.0:1787096277.738951:0:6477:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983920700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f9858421-28a9-4744-99eb-72e5bedc52c4+11:34143:x1873906077057792:12345-192.168.201.52@tcp:101:rm.0 Request processed in 233us (352us total) trans 4294967300 rc 301/301 00000100:00100000:0.0:1787096277.738957:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 157 00000100:00100000:3.0:1787096277.741245:0:3621:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:2.0:1787096277.741304:0:6480:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077057920 00000100:00100000:2.0:1787096277.741315:0:6480:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077057920 00000100:00100000:3.0:1787096277.741409:0:6479:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 22 00000100:00100000:3.0:1787096277.741425:0:6479:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7aac34d850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f9858421-28a9-4744-99eb-72e5bedc52c4+11:34143:x1873906077057920:12345-192.168.201.52@tcp:37:rm.0 00000100:00100000:0.0:1787096277.743318:0:6479:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7aac34d850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f9858421-28a9-4744-99eb-72e5bedc52c4+11:34143:x1873906077057920:12345-192.168.201.52@tcp:37:rm.0 Request processed in 1899us (2072us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096277.743326:0:6479:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 22 00000100:00100000:2.0:1787096277.745344:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:0.0:1787096277.745428:0:14257:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077058048 00000100:00100000:0.0:1787096277.745434:0:14257:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077058048 00000100:00100000:0.0:1787096277.745493:0:14257:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 23 00000100:00100000:0.0:1787096277.745506:0:14257:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983922300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f9858421-28a9-4744-99eb-72e5bedc52c4+11:34143:x1873906077058048:12345-192.168.201.52@tcp:37:rm.0 00000100:00100000:0.0:1787096277.747214:0:14257:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983922300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f9858421-28a9-4744-99eb-72e5bedc52c4+11:34143:x1873906077058048:12345-192.168.201.52@tcp:37:rm.0 Request processed in 1711us (1873us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096277.747220:0:14257:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 23 00000100:00100000:3.0:1787096277.748801:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:0.0:1787096277.748851:0:6479:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077058304 00000100:00100000:0.0:1787096277.748856:0:6479:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077058304 00000100:00100000:0.0:1787096277.748881:0:6479:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 24 00000100:00100000:0.0:1787096277.748888:0:6479:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7982f53480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f9858421-28a9-4744-99eb-72e5bedc52c4+11:34143:x1873906077058304:12345-192.168.201.52@tcp:35:rm.0 00000100:00100000:0.0:1787096277.748996:0:6479:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7982f53480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f9858421-28a9-4744-99eb-72e5bedc52c4+12:34143:x1873906077058304:12345-192.168.201.52@tcp:35:rm.0 Request processed in 107us (194us total) trans 4294967301 rc 0/0 00000100:00100000:0.0:1787096277.749005:0:6479:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 24 00000100:00100000:2.0:1787096277.750199:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:3.0:1787096277.750257:0:6476:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077058560 00000100:00100000:3.0:1787096277.750264:0:6476:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077058560 00000100:00100000:3.0:1787096277.750316:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 158 00000100:00100000:3.0:1787096277.750327:0:6476:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983922a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f9858421-28a9-4744-99eb-72e5bedc52c4+12:34143:x1873906077058560:12345-192.168.201.52@tcp:101:rm.0 00010000:00010000:3.0:1787096277.750341:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787096277.750448:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c8000/0x9eebb2eb7f50d76b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x4ef7bc65b7b410eb expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096277.750567:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9a00/0x9eebb2eb7f50d772 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096277.750580:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9a00/0x9eebb2eb7f50d772 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096277.750590:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9a00/0x9eebb2eb7f50d772 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787096277.750601:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x1:0x0]: returning size 4096 00010000:00010000:3.0:1787096277.750672:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c8000/0x9eebb2eb7f50d76b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410eb expref: 15 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096277.750690:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9a00/0x9eebb2eb7f50d772 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410eb expref: 14 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096277.750697:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff8c7a8b27af00) 00010000:00010000:3.0:1787096277.750701:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1787096277.750705:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7a869c9a00, rc 0) 00000100:00100000:3.0:1787096277.750743:0:6476:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983922a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f9858421-28a9-4744-99eb-72e5bedc52c4+14:34143:x1873906077058560:12345-192.168.201.52@tcp:101:rm.0 Request processed in 419us (550us total) trans 4294967302 rc 0/0 00000100:00100000:3.0:1787096277.750751:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 158 00000100:00100000:0.0:1787096277.752055:0:3621:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:2.0:1787096277.752109:0:21727:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077058688 00000100:00100000:2.0:1787096277.752120:0:21727:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077058688 00000100:00100000:2.0:1787096277.752182:0:21727:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 159 00000100:00100000:2.0:1787096277.752196:0:21727:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7aadf2c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:f9858421-28a9-4744-99eb-72e5bedc52c4+11:34143:x1873906077058688:12345-192.168.201.52@tcp:41:rm.0 00000004:00000024:2.0:1787096277.752257:0:21727:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 7847059456 avail 7384014848 00000100:00100000:2.0:1787096277.752333:0:21727:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7aadf2c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:f9858421-28a9-4744-99eb-72e5bedc52c4+11:34143:x1873906077058688:12345-192.168.201.52@tcp:41:rm.0 Request processed in 140us (276us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096277.752348:0:21727:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 159 00000100:00100000:0.0:1787096277.753338:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:2.0:1787096277.753414:0:8384:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077058816 00000100:00100000:2.0:1787096277.753419:0:8384:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077058816 00000100:00100000:2.0:1787096277.753456:0:8384:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 160 00000100:00100000:2.0:1787096277.753464:0:8384:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7aadf2f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f9858421-28a9-4744-99eb-72e5bedc52c4+11:34143:x1873906077058816:12345-192.168.201.52@tcp:101:rm.0 00010000:00010000:2.0:1787096277.753475:0:8384:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787096277.753494:0:8384:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766000/0x9eebb2eb7f50d780 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x4ef7bc65b7b410f2 expref: -99 pid: 8384 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753560:0:8384:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767e00/0x9eebb2eb7f50d787 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753571:0:8384:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767e00/0x9eebb2eb7f50d787 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753580:0:8384:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767e00/0x9eebb2eb7f50d787 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753602:0:8384:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d78e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753608:0:8384:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d78e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753613:0:8384:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d78e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753695:0:8384:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767c00/0x9eebb2eb7f50d795 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753705:0:8384:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767c00/0x9eebb2eb7f50d795 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753713:0:8384:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767c00/0x9eebb2eb7f50d795 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787096277.753744:0:8384:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x12:0x0]: returning size 0 00000004:00200000:2.0:1787096277.753747:0:8384:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787096277.753760:0:8384:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767c00/0x9eebb2eb7f50d795 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753768:0:8384:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767c00/0x9eebb2eb7f50d795 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753775:0:8384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767e00/0x9eebb2eb7f50d787 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753781:0:8384:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767e00/0x9eebb2eb7f50d787 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753797:0:8384:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767e00/0x9eebb2eb7f50d787 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753802:0:8384:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7a89767e00 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:2.0:1787096277.753806:0:8384:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767e00/0x9eebb2eb7f50d787 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753813:0:8384:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767e00/0x9eebb2eb7f50d787 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753827:0:8384:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2:0x0].0x0 (ffff8c7a8b27a100) 00010000:00010000:2.0:1787096277.753831:0:8384:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096277.753837:0:8384:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767e00/0x9eebb2eb7f50d787 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753844:0:8384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767e00/0x9eebb2eb7f50d787 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753858:0:8384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d78e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753865:0:8384:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d78e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753872:0:8384:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d78e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753877:0:8384:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7a89766600 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:2.0:1787096277.753881:0:8384:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d78e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753901:0:8384:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d78e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753912:0:8384:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2:0x0].0x7b711ae6 (ffff8c7a836ece00) 00010000:00010000:2.0:1787096277.753927:0:8384:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096277.753932:0:8384:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d78e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753939:0:8384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d78e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753973:0:8384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766000/0x9eebb2eb7f50d780 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410f2 expref: 13 pid: 8384 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753984:0:8384:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767c00/0x9eebb2eb7f50d795 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410f2 expref: 12 pid: 8384 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.753991:0:8384:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff8c7a836ecb00) 00010000:00010000:2.0:1787096277.753992:0:8384:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787096277.753995:0:8384:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7a89767c00, rc 0) 00000100:00100000:2.0:1787096277.754033:0:8384:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7aadf2f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f9858421-28a9-4744-99eb-72e5bedc52c4+12:34143:x1873906077058816:12345-192.168.201.52@tcp:101:rm.0 Request processed in 571us (697us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096277.754042:0:8384:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 160 00000100:00100000:0.0:1787096277.755139:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:2.0:1787096277.755217:0:10275:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077059072 00000100:00100000:2.0:1787096277.755233:0:10275:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077059072 00000100:00100000:2.0:1787096277.755275:0:10275:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 161 00000100:00100000:2.0:1787096277.755283:0:10275:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7aadf2d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f9858421-28a9-4744-99eb-72e5bedc52c4+12:34143:x1873906077059072:12345-192.168.201.52@tcp:36:rm.0 00010000:00010000:2.0:1787096277.755335:0:10275:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1787096277.755350:0:10275:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767c00/0x9eebb2eb7f50d795 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410f2 expref: 12 pid: 8384 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755388:0:10275:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff8c7a836ecb00) 00010000:00010000:2.0:1787096277.755426:0:10275:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983880400/0x9eebb2eb7f50d756 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410dd expref: 11 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755448:0:10275:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2:0x0].0x0 (ffff8c7a8b27a100) 00010000:00010000:2.0:1787096277.755457:0:10275:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787096277.755469:0:10275:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767000/0x9eebb2eb7f50d79c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755477:0:10275:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767000/0x9eebb2eb7f50d79c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755483:0:10275:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767000/0x9eebb2eb7f50d79c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755489:0:10275:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d7a3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755494:0:10275:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d7a3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755500:0:10275:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d7a3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755566:0:10275:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d7aa lrc: 3/0,1 mode: --/EX res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755582:0:10275:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d7aa lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755588:0:10275:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d7aa lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755764:0:10275:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813eca00/0x9eebb2eb7f50d7b1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755771:0:10275:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813eca00/0x9eebb2eb7f50d7b1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755776:0:10275:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813eca00/0x9eebb2eb7f50d7b1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755782:0:10275:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813eca00/0x9eebb2eb7f50d7b1 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755786:0:10275:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813eca00/0x9eebb2eb7f50d7b1 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755791:0:10275:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813eca00/0x9eebb2eb7f50d7b1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755812:0:10275:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1787096277.755826:0:10275:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d7aa lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755834:0:10275:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d7aa lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755841:0:10275:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d7aa lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.755846:0:6465:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787096277.755854:0:6465:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813eca00/0x9eebb2eb7f50d7b1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755860:0:10275:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7a813ed200 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:2.0:1787096277.755869:0:10275:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d7aa lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.755871:0:6465:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7a813eca00 already unused, calling callback (ffffffffc0bef6f0) 00010000:00010000:0.0:1787096277.755881:0:6465:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813eca00/0x9eebb2eb7f50d7b1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755884:0:10275:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d7aa lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755922:0:10275:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff8c7a836ecc00) 00010000:00010000:0.0:1787096277.755924:0:6465:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813eca00/0x9eebb2eb7f50d7b1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755927:0:10275:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787096277.755934:0:10275:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d7aa lrc: 2/0,0 mode: --/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755947:0:10275:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d7aa lrc: 0/0,0 mode: --/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.755952:0:6465:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff8c7a836ecc00) 00010000:00010000:0.0:1787096277.755955:0:6465:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787096277.755959:0:6465:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813eca00/0x9eebb2eb7f50d7b1 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.755981:0:6465:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813eca00/0x9eebb2eb7f50d7b1 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755985:0:10275:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767000/0x9eebb2eb7f50d79c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.755997:0:10275:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767000/0x9eebb2eb7f50d79c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.756005:0:10275:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767000/0x9eebb2eb7f50d79c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.756008:0:6465:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787096277.756014:0:10275:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7a89767000 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:0.0:1787096277.756014:0:6465:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096277.756022:0:6465:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787096277.756023:0:10275:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767000/0x9eebb2eb7f50d79c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.756036:0:10275:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767000/0x9eebb2eb7f50d79c lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.756060:0:10275:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2:0x0].0x0 (ffff8c7a836ec200) 00010000:00010000:2.0:1787096277.756063:0:10275:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096277.756066:0:10275:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767000/0x9eebb2eb7f50d79c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.756074:0:10275:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89767000/0x9eebb2eb7f50d79c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787096277.756091:0:10275:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000f8659260 reply state 00000000de7014f6 transno 4294967361 ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d7a3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.756102:0:10275:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d7a3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.756108:0:10275:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2:0x0].0x7b711ae6 (ffff8c7a836ece00) 00010000:00010000:2.0:1787096277.756110:0:10275:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787096277.756113:0:10275:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787096277.756169:0:10275:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7aadf2d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f9858421-28a9-4744-99eb-72e5bedc52c4+12:34143:x1873906077059072:12345-192.168.201.52@tcp:36:rm.0 Request processed in 886us (1029us total) trans 4294967361 rc 0/0 00000100:00100000:2.0:1787096277.756177:0:10275:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 161 00000100:00100000:0.0:1787096277.757621:0:3621:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:0.0:1787096277.757672:0:3621:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:3.0:1787096277.757702:0:6464:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077059200 00000100:00100000:0.0:1787096277.757702:0:14257:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077059456 00000100:00100000:0.0:1787096277.757708:0:14257:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077059456 00000100:00100000:3.0:1787096277.757710:0:6464:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077059200 00000100:00100000:0.0:1787096277.757745:0:14257:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 25 00000100:00100000:0.0:1787096277.757754:0:14257:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7aadf2e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f9858421-28a9-4744-99eb-72e5bedc52c4+14:34143:x1873906077059456:12345-192.168.201.52@tcp:35:rm.0 00000100:00100000:3.0:1787096277.757774:0:6464:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 4 00000100:00100000:3.0:1787096277.757785:0:6464:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7aadf2df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:f9858421-28a9-4744-99eb-72e5bedc52c4+12:2406:x1873906077059200:12345-192.168.201.52@tcp:103:rm.0 00010000:00010000:3.0:1787096277.757796:0:6464:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787096277.757814:0:6464:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983880800/0x9eebb2eb7f50d6ca lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410a5 expref: 12 pid: 10275 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096277.757827:0:6464:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff8c7a8b27a000) 00010000:00010000:3.0:1787096277.757837:0:6464:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787096277.757877:0:6464:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7aadf2df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:f9858421-28a9-4744-99eb-72e5bedc52c4+11:2406:x1873906077059200:12345-192.168.201.52@tcp:103:rm.0 Request processed in 94us (257us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096277.757884:0:14257:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7aadf2e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f9858421-28a9-4744-99eb-72e5bedc52c4+15:34143:x1873906077059456:12345-192.168.201.52@tcp:35:rm.0 Request processed in 126us (205us total) trans 4294967303 rc 0/0 00000100:00100000:3.0:1787096277.757888:0:6464:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 4 00000100:00100000:0.0:1787096277.757903:0:14257:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 25 00000100:00100000:0.0:1787096277.758979:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:2.0:1787096277.759051:0:6478:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077059712 00000100:00100000:2.0:1787096277.759060:0:6478:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077059712 00000100:00100000:2.0:1787096277.759128:0:6478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 162 00000100:00100000:2.0:1787096277.759141:0:6478:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7aadf2ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f9858421-28a9-4744-99eb-72e5bedc52c4+15:34143:x1873906077059712:12345-192.168.201.52@tcp:36:rm.0 00010000:00010000:2.0:1787096277.759184:0:6478:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787096277.759205:0:6478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9a00/0x9eebb2eb7f50d772 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410eb expref: 15 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.759251:0:6478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7983880000/0x9eebb2eb7f50d748 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410d6 expref: 14 pid: 10275 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.759269:0:6478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7983881c00/0x9eebb2eb7f50d6d8 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410ac expref: 13 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.759284:0:6478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff8c7a8b27af00) 00010000:00010000:2.0:1787096277.759289:0:6478:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787096277.759351:0:6478:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.759407:0:6478:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.759420:0:6478:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096277.759449:0:6478:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7983920380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6478:1873906084457088:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1787096277.759530:0:6478:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096277.759600:0:6482:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084457088 00000100:00100000:0.0:1787096277.759605:0:6482:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084457088 00000100:00100000:2.0:1787096277.759619:0:6478:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7aad283c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787096277.759647:0:6482:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 138 00000100:00100000:0.0:1787096277.759654:0:6482:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983923800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+5:6478:x1873906084457088:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787096277.759664:0:6482:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096277.759676:0:6482:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd762400/0x9eebb2eb7f50d7bf lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x9eebb2eb7f50d7b8 expref: -99 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.759693:0:6482:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750000/0x9eebb2eb7f50d6ae lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b4109e expref: 10 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.759712:0:6482:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd762400/0x9eebb2eb7f50d7bf lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d7b8 expref: 6 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.759738:0:6482:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750000/0x9eebb2eb7f50d6ae lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b4109e expref: 10 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.759748:0:6482:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750000/0x9eebb2eb7f50d6ae lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b4109e expref: 10 pid: 6476 timeout: 327 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096277.759776:0:6482:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7aadf2d500 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6482:1873906084457216:192.168.201.52@tcp:104: 00000100:00100000:0.0:1787096277.759855:0:6482:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7982067240 going to sleep for 11 seconds 00000100:00100000:0.0:1787096277.760618:0:6482:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7aadf2d500 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6482:1873906084457216:192.168.201.52@tcp:104: 00010000:00010000:0.0:1787096277.760636:0:6482:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd762400/0x9eebb2eb7f50d7bf lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d7b8 expref: 6 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.760643:0:6482:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8c7a86975c00) 00010000:00010000:0.0:1787096277.760645:0:6482:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096277.760648:0:6482:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd762400/0x9eebb2eb7f50d7bf lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d7b8 expref: 6 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.760654:0:6482:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7abd762400, rc 0) 00000100:00100000:0.0:1787096277.760694:0:6482:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983923800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:6478:x1873906084457088:12345-0@lo:101:mdt00_002.0 Request processed in 1041us (1184us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096277.760699:0:6482:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 138 00000100:00100000:3.0:1787096277.760730:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:2.0:1787096277.760748:0:6478:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7983920380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6478:1873906084457088:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787096277.760788:0:6478:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096277.760793:0:6462:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077059840 00010000:00010000:2.0:1787096277.760801:0:6478:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096277.760810:0:6462:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077059840 00010000:00010000:0.0:1787096277.760849:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750000/0x9eebb2eb7f50d6ae lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b4109e expref: 11 pid: 6476 timeout: 327 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096277.760886:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 57 00000100:00100000:0.0:1787096277.760897:0:6462:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7a90584c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f9858421-28a9-4744-99eb-72e5bedc52c4+11:2407:x1873906077059840:12345-192.168.201.52@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787096277.760907:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787096277.760911:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750000/0x9eebb2eb7f50d6ae lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b4109e expref: 11 pid: 6476 timeout: 327 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.760922:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750000/0x9eebb2eb7f50d6ae lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b4109e expref: 11 pid: 6476 timeout: 327 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.760935:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750000/0x9eebb2eb7f50d6ae lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b4109e expref: 11 pid: 6476 timeout: 327 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.760962:0:6462:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8c7a86975c00) 00010000:00010000:0.0:1787096277.760967:0:6462:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd762400/0x9eebb2eb7f50d7bf lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d7b8 expref: 5 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.760976:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd762400/0x9eebb2eb7f50d7bf lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x9eebb2eb7f50d7b8 expref: 5 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.760987:0:6462:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd762400/0x9eebb2eb7f50d7bf lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x9eebb2eb7f50d7b8 expref: 5 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.761015:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd762400/0x9eebb2eb7f50d7bf lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d7b8 expref: 5 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096277.761046:0:6462:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7aadf2d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6462:1873906084457344:0@lo:105: 00000100:00100000:0.0:1787096277.761095:0:6462:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096277.761141:0:6462:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7982067240 going to sleep for 11 seconds 00000100:00100000:2.0:1787096277.761160:0:6461:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084457344 00000100:00100000:2.0:1787096277.761167:0:6461:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084457344 00000100:00100000:2.0:1787096277.761227:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 37 00000100:00100000:2.0:1787096277.761241:0:6461:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7aadf2ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6462:x1873906084457344:12345-0@lo:105: 00010000:00010000:2.0:1787096277.761255:0:6461:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.761267:0:6461:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.761277:0:6461:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.761288:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.761296:0:6461:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.761308:0:6461:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.761340:0:6461:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff8c7a813ec400) 00010000:00010000:3.0:1787096277.761405:0:6478:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096277.761438:0:6478:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787096277.761461:0:6478:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1787096277.761467:0:6461:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7aadf2ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6462:x1873906084457344:12345-0@lo:105: Request processed in 226us (366us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096277.761473:0:6462:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7aadf2d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6462:1873906084457344:0@lo:105: 00010000:00010000:3.0:1787096277.761489:0:6478:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c8000/0x9eebb2eb7f50d7c6 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096277.761497:0:6478:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c8000/0x9eebb2eb7f50d7c6 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096277.761498:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 37 00010000:00010000:0.0:1787096277.761501:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787096277.761506:0:6478:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c8000/0x9eebb2eb7f50d7c6 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096277.761546:0:6478:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7982f51880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6478:1873906084457472:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787096277.761552:0:6462:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7a90584c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f9858421-28a9-4744-99eb-72e5bedc52c4+10:2407:x1873906077059840:12345-192.168.201.52@tcp:103:ldlm_bl.0 Request processed in 653us (823us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096277.761559:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 57 00000100:00100000:3.0:1787096277.761581:0:6478:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096277.761604:0:6478:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7aa07c8b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787096277.761637:0:6481:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084457472 00000100:00100000:2.0:1787096277.761644:0:6481:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084457472 00000100:00100000:2.0:1787096277.761701:0:6481:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 139 00000100:00100000:2.0:1787096277.761721:0:6481:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7982f51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:6478:x1873906084457472:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787096277.761734:0:6481:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787096277.761764:0:6481:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed400/0x9eebb2eb7f50d7cd lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x9eebb2eb7f50d7c6 expref: -99 pid: 6481 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.761788:0:6481:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed400/0x9eebb2eb7f50d7cd lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d7c6 expref: 7 pid: 6481 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.761804:0:6481:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed400/0x9eebb2eb7f50d7cd lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d7c6 expref: 7 pid: 6481 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.761813:0:6481:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa5d0027 (ffff8c7a836ec400) 00010000:00010000:2.0:1787096277.761817:0:6481:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096277.761820:0:6481:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7a813ed400, rc 0) 00000100:00100000:2.0:1787096277.761886:0:6481:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7982f51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6478:x1873906084457472:12345-0@lo:101:mdt00_002.0 Request processed in 171us (301us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096277.761894:0:6481:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 139 00000100:00100000:3.0:1787096277.761903:0:6478:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7982f51880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6478:1873906084457472:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787096277.761922:0:6478:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c8000/0x9eebb2eb7f50d7c6 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x9eebb2eb7f50d7cd expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096277.761934:0:6478:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c8000/0x9eebb2eb7f50d7c6 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d7cd expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787096277.761948:0:6478:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1787096277.761980:0:6478:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7982f51880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6478:1873906084457600:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787096277.762015:0:6478:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096277.762041:0:6478:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7aa07c8d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787096277.762067:0:7980:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084457600 00000100:00100000:2.0:1787096277.762084:0:7980:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084457600 00000100:00100000:2.0:1787096277.762130:0:7980:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 140 00000100:00100000:2.0:1787096277.762150:0:7980:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7982f52680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:6478:x1873906084457600:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787096277.762237:0:7980:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7982f52680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:6478:x1873906084457600:12345-0@lo:1000:mdt00_002.0 Request processed in 97us (222us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096277.762247:0:7980:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 140 00000100:00100000:0.0:1787096277.762277:0:6478:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7982f51880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6478:1873906084457600:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787096277.762314:0:6478:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7aadf2df80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6478:1873906084457728:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787096277.762370:0:6478:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096277.762395:0:6478:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7982067180 going to sleep for 11 seconds 00000100:00100000:1.0:1787096277.762419:0:7407:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084457728 00000100:00100000:1.0:1787096277.762426:0:7407:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084457728 00000100:00100000:1.0:1787096277.762472:0:7407:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 141 00000100:00100000:1.0:1787096277.762480:0:7407:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7aadf2d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6478:x1873906084457728:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787096277.762571:0:7407:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7aadf2d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6478:x1873906084457728:12345-0@lo:1000:mdt00_002.0 Request processed in 92us (217us total) trans 0 rc 1/1 00000100:00100000:1.0:1787096277.762578:0:7407:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 141 00000100:00100000:0.0:1787096277.762598:0:6478:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7aadf2df80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6478:1873906084457728:0@lo:1000:mdt00_002.0 00010000:00010000:0.0:1787096277.762635:0:6478:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762800/0x9eebb2eb7f50d7d4 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.762642:0:6478:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762800/0x9eebb2eb7f50d7d4 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.762648:0:6478:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762800/0x9eebb2eb7f50d7d4 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096277.762657:0:6478:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7aadf2df80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6478:1873906084457856:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787096277.762680:0:6478:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096277.762697:0:6478:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c79831d7a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787096277.762700:0:6482:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084457856 00000100:00100000:1.0:1787096277.762704:0:6482:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084457856 00000100:00100000:1.0:1787096277.762730:0:6482:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 142 00000100:00100000:1.0:1787096277.762736:0:6482:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7aadf2dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:6478:x1873906084457856:12345-0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787096277.762743:0:6482:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787096277.762756:0:6482:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983880a00/0x9eebb2eb7f50d7db lrc: 2/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x9eebb2eb7f50d7d4 expref: -99 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.762782:0:6482:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983880a00/0x9eebb2eb7f50d7db lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d7d4 expref: 8 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.762791:0:6482:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983880a00/0x9eebb2eb7f50d7db lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d7d4 expref: 8 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.762800:0:6482:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff8c7a8b27ad00) 00010000:00010000:1.0:1787096277.762803:0:6482:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787096277.762806:0:6482:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7983880a00, rc 0) 00000100:00100000:1.0:1787096277.762849:0:6482:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7aadf2dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6478:x1873906084457856:12345-0@lo:101:mdt00_002.0 Request processed in 112us (167us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096277.762858:0:6482:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 142 00000100:00100000:0.0:1787096277.762872:0:6478:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7aadf2df80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6478:1873906084457856:0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787096277.762892:0:6478:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762800/0x9eebb2eb7f50d7d4 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x9eebb2eb7f50d7db expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.762899:0:6478:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762800/0x9eebb2eb7f50d7d4 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d7db expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.762916:0:6478:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d7e2 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.762923:0:6478:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d7e2 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.762929:0:6478:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d7e2 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.762940:0:6478:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762000/0x9eebb2eb7f50d7e9 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.762948:0:6478:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7aa2750200/0x9eebb2eb7f50d6e6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410b3 expref: 12 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.762954:0:6478:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762000/0x9eebb2eb7f50d7e9 lrc: 4/0,1 mode: --/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.762967:0:6478:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff8c7aa2750200/0x9eebb2eb7f50d6e6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410b3 expref: 12 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.762976:0:6478:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7aa2750200/0x9eebb2eb7f50d6e6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410b3 expref: 12 pid: 6476 timeout: 327 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096277.762985:0:6478:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7aadf2df80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6478:1873906084457984:192.168.201.52@tcp:104: 00000100:00100000:0.0:1787096277.763035:0:6478:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c79831d7a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787096277.764033:0:6478:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7aadf2df80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6478:1873906084457984:192.168.201.52@tcp:104: 00010000:00010000:1.0:1787096277.764066:0:6478:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762000/0x9eebb2eb7f50d7e9 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096277.764809:0:3621:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:0.0:1787096277.764852:0:7401:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077059968 00000100:00100000:0.0:1787096277.764859:0:7401:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077059968 00010000:00010000:0.0:1787096277.764880:0:7401:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff8c7aa2750200/0x9eebb2eb7f50d6e6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410b3 expref: 13 pid: 6476 timeout: 327 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096277.764907:0:7401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 58 00000100:00100000:0.0:1787096277.764916:0:7401:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7a879f3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f9858421-28a9-4744-99eb-72e5bedc52c4+13:2407:x1873906077059968:12345-192.168.201.52@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787096277.764924:0:7401:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787096277.764929:0:7401:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff8c7aa2750200/0x9eebb2eb7f50d6e6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410b3 expref: 13 pid: 6476 timeout: 327 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.764940:0:7401:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff8c7aa2750200/0x9eebb2eb7f50d6e6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410b3 expref: 13 pid: 6476 timeout: 327 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.764954:0:7401:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7aa2750200/0x9eebb2eb7f50d6e6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x2:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410b3 expref: 13 pid: 6476 timeout: 327 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.764964:0:7401:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2:0x0].0x0 (ffff8c7a8b27a700) 00010000:00010000:0.0:1787096277.764967:0:7401:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762000/0x9eebb2eb7f50d7e9 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.764973:0:7401:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762000/0x9eebb2eb7f50d7e9 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.764978:0:7401:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762000/0x9eebb2eb7f50d7e9 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.764998:0:7401:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787096277.765020:0:6478:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762000/0x9eebb2eb7f50d7e9 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096277.765032:0:7401:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7a879f3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f9858421-28a9-4744-99eb-72e5bedc52c4+12:2407:x1873906077059968:12345-192.168.201.52@tcp:103:ldlm_bl.0 Request processed in 117us (223us total) trans 0 rc 0/0 00010000:00010000:2.0:1787096277.765039:0:6478:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762000/0x9eebb2eb7f50d7e9 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096277.765044:0:7401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 58 00010000:00010000:2.0:1787096277.765075:0:6478:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ed200/0x9eebb2eb7f50d7f0 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.765083:0:6478:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ed200/0x9eebb2eb7f50d7f0 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.765093:0:6478:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ed200/0x9eebb2eb7f50d7f0 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096277.765110:0:6478:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7983920000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6478:1873906084458112:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1787096277.765142:0:6478:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096277.765160:0:6478:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7aad283c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787096277.765169:0:6481:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084458112 00000100:00100000:0.0:1787096277.765174:0:6481:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084458112 00000100:00100000:0.0:1787096277.765209:0:6481:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 143 00000100:00100000:0.0:1787096277.765218:0:6481:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983920a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6478:x1873906084458112:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787096277.765227:0:6481:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096277.765238:0:6481:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763400/0x9eebb2eb7f50d7f7 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x9eebb2eb7f50d7f0 expref: -99 pid: 6481 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.765252:0:6481:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763400/0x9eebb2eb7f50d7f7 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d7f0 expref: 9 pid: 6481 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.765261:0:6481:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763400/0x9eebb2eb7f50d7f7 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d7f0 expref: 9 pid: 6481 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.765266:0:6481:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2:0x0].0x0 (ffff8c7a92c71b00) 00010000:00010000:0.0:1787096277.765268:0:6481:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787096277.765270:0:6481:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7abd763400, rc 0) 00000100:00100000:0.0:1787096277.765299:0:6481:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983920a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6478:x1873906084458112:12345-0@lo:101:mdt00_002.0 Request processed in 84us (157us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096277.765304:0:6481:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 143 00000100:00100000:2.0:1787096277.767702:0:6478:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7983920000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6478:1873906084458112:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787096277.767714:0:6478:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ed200/0x9eebb2eb7f50d7f0 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x9eebb2eb7f50d7f7 expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.767723:0:6478:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ed200/0x9eebb2eb7f50d7f0 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d7f7 expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096277.768311:0:6478:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7983920000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6478:1873906084458240:0@lo:403:mdt00_002.0 00000100:00100000:2.0:1787096277.768379:0:6478:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096277.768400:0:6478:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c79831ab780 going to sleep for 11 seconds 00000100:00100000:2.0:1787096277.768415:0:7980:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084458240 00000100:00100000:2.0:1787096277.768419:0:7980:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084458240 00000100:00100000:2.0:1787096277.768442:0:7980:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 144 00000100:00100000:2.0:1787096277.768456:0:7980:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983920380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:6478:x1873906084458240:12345-0@lo:403:mdt00_002.0 00000100:00100000:2.0:1787096277.770189:0:7980:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983920380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:6478:x1873906084458240:12345-0@lo:403:mdt00_002.0 Request processed in 1734us (1808us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096277.770195:0:7980:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 144 00000100:00100000:0.0:1787096277.770219:0:6478:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7983920000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6478:1873906084458240:0@lo:403:mdt00_002.0 00000040:00080000:0.0:1787096277.771014:0:6478:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].2, 936 off35856 00000100:00100000:0.0:1787096277.771179:0:6478:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7aadf2f100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6478:1873906084458368:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787096277.771216:0:6478:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096277.771245:0:6478:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7983307240 going to sleep for 11 seconds 00000100:00100000:1.0:1787096277.771263:0:7407:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084458368 00000100:00100000:1.0:1787096277.771269:0:7407:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084458368 00000100:00100000:1.0:1787096277.771299:0:7407:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 145 00000100:00100000:1.0:1787096277.771305:0:7407:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7aadf2ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6478:x1873906084458368:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787096277.771386:0:7407:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7aadf2ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6478:x1873906084458368:12345-0@lo:1000:mdt00_002.0 Request processed in 81us (169us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096277.771393:0:7407:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 145 00000100:00100000:0.0:1787096277.771413:0:6478:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7aadf2f100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6478:1873906084458368:0@lo:1000:mdt00_002.0 00000040:00080000:0.0:1787096277.771435:0:6478:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].2, 1016 off36136 00000004:00080000:0.0:1787096277.771442:0:6478:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff8c79831d7600 with version 5 00000004:00080000:1.0:1787096277.771463:0:7405:0:(osp_trans.c:1330:osp_get_next_request()) ou ffff8c7aad23da80 version 5 rpc_version 5 00000020:00100000:1.0:1787096277.771481:0:7405:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787096277.771492:0:7405:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8c7a879f0e00 x1873906084458624/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2592/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787096277.771504:0:7405:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7a879f0e00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7405:1873906084458624:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787096277.771540:0:7405:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096277.771561:0:7405:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7abf92c180 going to sleep for 11 seconds 00000100:00100000:0.0:1787096277.771576:0:6482:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084458624 00000100:00100000:0.0:1787096277.771581:0:6482:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084458624 00000100:00100000:0.0:1787096277.771615:0:6482:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 146 00000100:00100000:0.0:1787096277.771622:0:6482:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7a879f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:7405:x1873906084458624:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787096277.771832:0:6482:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7a879f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7405:x1873906084458624:12345-0@lo:1000:osp_up0-1.0 Request processed in 211us (290us total) trans 4294967362 rc 0/0 00000100:00100000:0.0:1787096277.771838:0:6482:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 146 00000100:00100000:1.0:1787096277.771873:0:7405:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7a879f0e00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7405:1873906084458624:0@lo:1000:osp_up0-1.0 00000004:00200000:0.0:1787096277.771888:0:6478:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x1:0x0]: returning size 4096 00010000:00010000:0.0:1787096277.771902:0:6478:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762000/0x9eebb2eb7f50d7e9 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.771908:0:6478:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762000/0x9eebb2eb7f50d7e9 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.771913:0:6478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762000/0x9eebb2eb7f50d7e9 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.771917:0:6478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7abd762000 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:0.0:1787096277.771920:0:6478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762000/0x9eebb2eb7f50d7e9 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.771925:0:6478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762000/0x9eebb2eb7f50d7e9 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.771934:0:6478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2:0x0].0x0 (ffff8c7a8b27a700) 00010000:00010000:0.0:1787096277.771936:0:6478:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787096277.771945:0:6478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762000/0x9eebb2eb7f50d7e9 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.771949:0:6478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762000/0x9eebb2eb7f50d7e9 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.771957:0:6478:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ed200/0x9eebb2eb7f50d7f0 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d7f7 expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.771962:0:6478:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ed200/0x9eebb2eb7f50d7f0 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x9eebb2eb7f50d7f7 expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.771966:0:6478:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ed200/0x9eebb2eb7f50d7f0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x9eebb2eb7f50d7f7 expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.771982:0:6478:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787096277.771987:0:6478:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000401:0x2:0x0] 00010000:00010000:0.0:1787096277.771992:0:6478:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d7e2 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.771996:0:6478:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d7e2 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.771999:0:6478:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d7e2 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.772003:0:6478:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7abd763a00 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:1.0:1787096277.772004:0:8374:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787096277.772006:0:6478:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d7e2 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.772013:0:6478:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d7e2 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.772014:0:8374:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ed200/0x9eebb2eb7f50d7f0 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x9eebb2eb7f50d7f7 expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.772035:0:8374:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7a813ed200 already unused, calling callback (ffffffffc1765ee0) 00010000:00010000:0.0:1787096277.772039:0:6478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff8c7a8b27af00) 00010000:00010000:1.0:1787096277.772041:0:8374:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ed200/0x9eebb2eb7f50d7f0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x9eebb2eb7f50d7f7 expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.772044:0:6478:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096277.772048:0:6478:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d7e2 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787096277.772054:0:8374:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ed200/0x9eebb2eb7f50d7f0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x9eebb2eb7f50d7f7 expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.772055:0:6478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d7e2 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.772078:0:6478:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762800/0x9eebb2eb7f50d7d4 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d7db expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.772083:0:6478:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762800/0x9eebb2eb7f50d7d4 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x9eebb2eb7f50d7db expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.772088:0:6478:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762800/0x9eebb2eb7f50d7d4 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x9eebb2eb7f50d7db expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.772091:0:8374:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ed200/0x9eebb2eb7f50d7f0 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x9eebb2eb7f50d7f7 expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.772097:0:8374:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787096277.772104:0:6478:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787096277.772112:0:6478:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787096277.772115:0:8374:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ed200/0x9eebb2eb7f50d7f0 lrc: 1/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x9eebb2eb7f50d7f7 expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.772115:0:6478:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.772137:0:7400:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787096277.772139:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ed200/0x9eebb2eb7f50d7f0 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x9eebb2eb7f50d7f7 expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.772140:0:6478:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.772145:0:7400:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762800/0x9eebb2eb7f50d7d4 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x9eebb2eb7f50d7db expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.772167:0:6478:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.772168:0:7400:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7abd762800 already unused, calling callback (ffffffffc1765ee0) 00010000:00010000:1.0:1787096277.772171:0:8374:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787096277.772175:0:7400:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762800/0x9eebb2eb7f50d7d4 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x9eebb2eb7f50d7db expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.772180:0:8374:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787096277.772186:0:8374:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000004:00010000:2.0:1787096277.772188:0:7400:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762800/0x9eebb2eb7f50d7d4 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x9eebb2eb7f50d7db expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.772193:0:6478:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:1.0:1787096277.772197:0:8374:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.772209:0:8374:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7a813ec400 already unused, calling callback (ffffffffc1765ee0) 00000004:00080000:0.0:1787096277.772209:0:6478:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787096277.772210:0:7400:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762800/0x9eebb2eb7f50d7d4 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x9eebb2eb7f50d7db expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.772217:0:8374:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.772218:0:7400:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787096277.772220:0:6466:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00010000:1.0:1787096277.772229:0:8374:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.772230:0:6478:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c8000/0x9eebb2eb7f50d7c6 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x9eebb2eb7f50d7cd expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.772239:0:7400:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762800/0x9eebb2eb7f50d7d4 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x9eebb2eb7f50d7db expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.772255:0:6478:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c8000/0x9eebb2eb7f50d7c6 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x9eebb2eb7f50d7cd expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.772256:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762800/0x9eebb2eb7f50d7d4 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x9eebb2eb7f50d7db expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.772259:0:8374:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.772279:0:8374:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787096277.772289:0:8374:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.772290:0:7400:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787096277.772293:0:7400:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787096277.772297:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d7b8 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x9eebb2eb7f50d7bf expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096277.772298:0:7400:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787096277.772310:0:3628:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7983923b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3628:1873906084458880:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787096277.772314:0:8374:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787096277.772316:0:6478:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7aadf2ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f9858421-28a9-4744-99eb-72e5bedc52c4+12:34143:x1873906077059712:12345-192.168.201.52@tcp:36:rm.0 Request processed in 13178us (13336us total) trans 4294967304 rc 0/0 00010000:00010000:1.0:1787096277.772317:0:8374:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787096277.772321:0:8374:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787096277.772328:0:6478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 162 00000100:00100000:1.0:1787096277.772336:0:3627:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7983909f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3627:1873906084458752:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787096277.772877:0:3628:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096277.772907:0:10225:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084458880 00000100:00100000:0.0:1787096277.772911:0:10225:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084458880 00000100:00100000:2.0:1787096277.772927:0:3628:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787096277.772932:0:3627:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096277.772938:0:3628:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7983386680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3628:1873906084459008:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787096277.772950:0:10225:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084458752 00000100:00100000:0.0:1787096277.772953:0:10225:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084458752 00000100:00100000:2.0:1787096277.772970:0:3628:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096277.772983:0:10225:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084459008 00000100:00100000:0.0:1787096277.772986:0:10225:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084459008 00000100:00100000:1.0:1787096277.773005:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5 00000100:00100000:0.0:1787096277.773021:0:10225:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6 00000100:00100000:1.0:1787096277.773022:0:6463:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983921500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+12:3628:x1873906084458880:12345-0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787096277.773032:0:10225:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983a68a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+12:3627:x1873906084458752:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787096277.773045:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787096277.773046:0:10225:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787096277.773063:0:10225:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763400/0x9eebb2eb7f50d7f7 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x9eebb2eb7f50d7f0 expref: 12 pid: 6481 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096277.773065:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983880a00/0x9eebb2eb7f50d7db lrc: 0/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x9eebb2eb7f50d7d4 expref: 12 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.773085:0:10225:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2:0x0].0x0 (ffff8c7a92c71b00) 00010000:00010000:1.0:1787096277.773087:0:6463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff8c7a8b27ad00) 00010000:00010000:0.0:1787096277.773096:0:10225:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787096277.773100:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787096277.773135:0:10225:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983a68a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+10:3627:x1873906084458752:12345-0@lo:103:ldlm_bl.0 Request processed in 106us (205us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096277.773141:0:10225:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6 00000100:00100000:1.0:1787096277.773146:0:6463:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983921500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+10:3628:x1873906084458880:12345-0@lo:103:ldlm_bl.0 Request processed in 129us (786us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096277.773154:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5 00000100:00100000:0.0:1787096277.773169:0:10225:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7 00000100:00100000:0.0:1787096277.773186:0:10225:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983920000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+8:3628:x1873906084459008:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787096277.773196:0:10225:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787096277.773204:0:3627:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7983909f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3627:1873906084458752:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787096277.773212:0:10225:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd762400/0x9eebb2eb7f50d7bf lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x9eebb2eb7f50d7b8 expref: 8 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096277.773224:0:10225:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8c7a86975c00) 00010000:00010000:0.0:1787096277.773234:0:10225:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787096277.773266:0:10225:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983920000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+7:3628:x1873906084459008:12345-0@lo:103:ldlm_bl.0 Request processed in 83us (297us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096277.773270:0:10225:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7 00000100:00100000:2.0:1787096277.773399:0:3628:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7983386680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3628:1873906084459008:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787096277.773406:0:3628:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7983923b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3628:1873906084458880:0@lo:103:ldlm_bl.0 00000001:02000400:1.0:1787096278.329072:0:21961:0:(debug.c:651:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-ec test 23b: truncate EC file to larger size === 19:37:57 (1787096277) 00000100:00100000:1.0:1787096278.411227:0:3621:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:0.0:1787096278.411279:0:6477:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077060096 00000100:00100000:0.0:1787096278.411288:0:6477:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077060096 00000100:00080000:3.0:1787096278.411339:0:6473:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787096278.411348:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 163 00000100:00100000:0.0:1787096278.411394:0:6477:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983a69c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f9858421-28a9-4744-99eb-72e5bedc52c4+10:34319:x1873906077060096:12345-192.168.201.52@tcp:101:lfs.0 00010000:00010000:0.0:1787096278.411418:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096278.411437:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763400/0x9eebb2eb7f50d7fe lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x4ef7bc65b7b410f9 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.411492:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d805 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.411516:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d805 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.411528:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d805 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787096278.411567:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:0.0:1787096278.411570:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787096278.411596:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d805 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.411616:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763400/0x9eebb2eb7f50d7fe lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410f9 expref: 12 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.411629:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d805 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410f9 expref: 11 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.411636:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8c7a92c71b00) 00010000:00010000:0.0:1787096278.411638:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787096278.411642:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7abd763a00, rc 0) 00000100:00100000:0.0:1787096278.411694:0:6477:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983a69c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f9858421-28a9-4744-99eb-72e5bedc52c4+11:34319:x1873906077060096:12345-192.168.201.52@tcp:101:lfs.0 Request processed in 308us (474us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096278.411700:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 163 00000100:00100000:0.0:1787096278.412819:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:3.0:1787096278.412855:0:6476:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077060224 00000100:00100000:3.0:1787096278.412865:0:6476:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077060224 00000100:00100000:3.0:1787096278.412911:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 164 00000100:00100000:3.0:1787096278.412919:0:6476:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7aadf2f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f9858421-28a9-4744-99eb-72e5bedc52c4+11:34319:x1873906077060224:12345-192.168.201.52@tcp:101:lfs.0 00010000:00010000:3.0:1787096278.412935:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787096278.412950:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9e00/0x9eebb2eb7f50d80c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x4ef7bc65b7b41100 expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.412999:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9400/0x9eebb2eb7f50d813 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413011:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9400/0x9eebb2eb7f50d813 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413019:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9400/0x9eebb2eb7f50d813 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413042:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d81a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413047:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d81a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413051:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d81a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413092:0:6476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9400/0x9eebb2eb7f50d813 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413097:0:6476:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9400/0x9eebb2eb7f50d813 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413117:0:6476:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9400/0x9eebb2eb7f50d813 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413122:0:6476:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7a869c9400 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:3.0:1787096278.413125:0:6476:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9400/0x9eebb2eb7f50d813 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413132:0:6476:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9400/0x9eebb2eb7f50d813 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413142:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8c7a92c71b00) 00010000:00010000:3.0:1787096278.413144:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787096278.413148:0:6476:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9400/0x9eebb2eb7f50d813 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413153:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9400/0x9eebb2eb7f50d813 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413161:0:6476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d81a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413165:0:6476:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d81a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413169:0:6476:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d81a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413172:0:6476:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7a869c9800 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:3.0:1787096278.413174:0:6476:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d81a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413193:0:6476:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d81a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413198:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9280d55e (ffff8c7aadca5c00) 00010000:00010000:3.0:1787096278.413200:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787096278.413201:0:6476:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d81a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413205:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d81a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413229:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9e00/0x9eebb2eb7f50d80c lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41100 expref: 12 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413236:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8c7a92c71b00) 00010000:00010000:3.0:1787096278.413238:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787096278.413251:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a869c9e00/0x9eebb2eb7f50d80c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41100 expref: 12 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.413257:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7a869c9e00, rc 0) 00000100:00100000:3.0:1787096278.413325:0:6476:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7aadf2f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f9858421-28a9-4744-99eb-72e5bedc52c4+11:34319:x1873906077060224:12345-192.168.201.52@tcp:101:lfs.0 Request processed in 396us (495us total) trans 0 rc 301/301 00000100:00100000:3.0:1787096278.413334:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 164 00000100:00100000:2.0:1787096278.415547:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:3.0:1787096278.415597:0:21727:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077060480 00000100:00100000:3.0:1787096278.415611:0:21727:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077060480 00000100:00080000:2.0:1787096278.415660:0:6473:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787096278.415687:0:21727:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 165 00000100:00100000:3.0:1787096278.415698:0:21727:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983923b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:f9858421-28a9-4744-99eb-72e5bedc52c4+12:34319:x1873906077060480:12345-192.168.201.52@tcp:36:lfs.0 00000100:00100000:3.0:1787096278.415861:0:21727:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7982f50e00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084459136:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787096278.415930:0:21727:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096278.415982:0:6481:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084459136 00000100:00100000:3.0:1787096278.415983:0:21727:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7aa07c8240 going to sleep for 11 seconds 00000100:00100000:0.0:1787096278.415994:0:6481:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084459136 00000100:00100000:0.0:1787096278.416039:0:6481:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 147 00000100:00100000:0.0:1787096278.416046:0:6481:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7982f51500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:21727:x1873906084459136:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787096278.416118:0:6481:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7982f51500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:21727:x1873906084459136:12345-0@lo:1000:mdt00_005.0 Request processed in 73us (186us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096278.416125:0:6481:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 147 00000100:00100000:3.0:1787096278.416162:0:21727:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7982f50e00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084459136:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787096278.416205:0:21727:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7982f50e00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084459264:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787096278.416227:0:21727:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096278.416246:0:21727:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7aa07c8240 going to sleep for 11 seconds 00000100:00100000:2.0:1787096278.416272:0:7980:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084459264 00000100:00100000:2.0:1787096278.416279:0:7980:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084459264 00000100:00100000:2.0:1787096278.416310:0:7980:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 148 00000100:00100000:2.0:1787096278.416316:0:7980:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7982f50a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:21727:x1873906084459264:12345-0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1787096278.416424:0:7980:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7982f50a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:21727:x1873906084459264:12345-0@lo:1000:mdt00_005.0 Request processed in 108us (196us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787096278.416430:0:7980:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 148 00000100:00100000:0.0:1787096278.416472:0:21727:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7982f50e00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084459264:0@lo:1000:mdt00_005.0 00010000:00010000:0.0:1787096278.416548:0:21727:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 21727 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.416559:0:21727:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.416579:0:21727:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096278.416613:0:21727:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7aadf2d180 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084459392:0@lo:101:mdt00_005.0 00000100:00100000:0.0:1787096278.416651:0:21727:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096278.416686:0:21727:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7983307900 going to sleep for 11 seconds 00000100:00100000:1.0:1787096278.416715:0:7407:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084459392 00000100:00100000:1.0:1787096278.416720:0:7407:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084459392 00000100:00100000:1.0:1787096278.416749:0:7407:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 149 00000100:00100000:1.0:1787096278.416755:0:7407:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7aadf2fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:21727:x1873906084459392:12345-0@lo:101:mdt00_005.0 00010000:00010000:1.0:1787096278.416764:0:7407:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787096278.416775:0:7407:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983881800/0x9eebb2eb7f50d828 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x9eebb2eb7f50d821 expref: -99 pid: 7407 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.416794:0:7407:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d805 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410f9 expref: 10 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.416802:0:7407:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983881800/0x9eebb2eb7f50d828 lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d821 expref: 8 pid: 7407 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.416821:0:7407:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d805 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410f9 expref: 10 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.416837:0:7407:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d805 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410f9 expref: 10 pid: 6477 timeout: 328 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096278.416846:0:7407:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7983a6b100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:7407:1873906084459520:192.168.201.52@tcp:104: 00000100:00100000:1.0:1787096278.416896:0:7407:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7abf92c9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787096278.417833:0:7407:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7983a6b100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:7407:1873906084459520:192.168.201.52@tcp:104: 00010000:00010000:3.0:1787096278.417863:0:7407:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983881800/0x9eebb2eb7f50d828 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d821 expref: 8 pid: 7407 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.417874:0:7407:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8c7a92c71b00) 00010000:00010000:3.0:1787096278.417926:0:7407:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787096278.417930:0:7407:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983881800/0x9eebb2eb7f50d828 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d821 expref: 8 pid: 7407 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.417939:0:7407:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7983881800, rc 0) 00000100:00100000:3.0:1787096278.417986:0:7407:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7aadf2fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:21727:x1873906084459392:12345-0@lo:101:mdt00_005.0 Request processed in 1230us (1334us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096278.417999:0:7407:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 149 00000100:00100000:2.0:1787096278.418025:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:0.0:1787096278.418035:0:21727:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7aadf2d180 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084459392:0@lo:101:mdt00_005.0 00010000:00010000:0.0:1787096278.418054:0:21727:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.418066:0:21727:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096278.418069:0:6463:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077060608 00000100:00100000:1.0:1787096278.418075:0:6463:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077060608 00010000:00010000:1.0:1787096278.418099:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d805 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410f9 expref: 11 pid: 6477 timeout: 328 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096278.418122:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 59 00000100:00100000:1.0:1787096278.418128:0:6463:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983386680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f9858421-28a9-4744-99eb-72e5bedc52c4+11:2407:x1873906077060608:12345-192.168.201.52@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787096278.418148:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787096278.418153:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d805 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410f9 expref: 11 pid: 6477 timeout: 328 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.418161:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d805 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410f9 expref: 11 pid: 6477 timeout: 328 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.418174:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763a00/0x9eebb2eb7f50d805 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b410f9 expref: 11 pid: 6477 timeout: 328 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.418185:0:6463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8c7a92c71b00) 00010000:00010000:1.0:1787096278.418188:0:6463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983881800/0x9eebb2eb7f50d828 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d821 expref: 7 pid: 7407 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.418204:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983881800/0x9eebb2eb7f50d828 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x9eebb2eb7f50d821 expref: 7 pid: 7407 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.418210:0:6463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983881800/0x9eebb2eb7f50d828 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x9eebb2eb7f50d821 expref: 7 pid: 7407 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.418226:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983881800/0x9eebb2eb7f50d828 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d821 expref: 7 pid: 7407 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096278.418234:0:6463:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7983a6ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6463:1873906084459648:0@lo:105: 00000100:00100000:1.0:1787096278.418270:0:6463:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096278.418291:0:6463:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7abf92c180 going to sleep for 11 seconds 00000100:00100000:0.0:1787096278.418310:0:8373:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084459648 00000100:00100000:0.0:1787096278.418315:0:8373:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084459648 00000100:00100000:0.0:1787096278.418382:0:8373:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 38 00000100:00100000:0.0:1787096278.418393:0:8373:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983a69180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:lustre-MDT0001-mdtlov_UUID+4:6463:x1873906084459648:12345-0@lo:105: 00010000:00010000:0.0:1787096278.418405:0:8373:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.418416:0:8373:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.418425:0:8373:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.418433:0:8373:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.418447:0:8373:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.418457:0:8373:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.418480:0:8373:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff8c7abd762000) 00010000:00010000:3.0:1787096278.418511:0:21727:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.418526:0:21727:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096278.418538:0:8373:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983a69180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:lustre-MDT0001-mdtlov_UUID+4:6463:x1873906084459648:12345-0@lo:105: Request processed in 146us (264us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096278.418556:0:6463:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7983a6ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6463:1873906084459648:0@lo:105: 00000100:00100000:0.0:1787096278.418562:0:8373:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 38 00010000:00010000:1.0:1787096278.418580:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:3.0:1787096278.418594:0:21727:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:1.0:1787096278.418604:0:6463:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983386680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f9858421-28a9-4744-99eb-72e5bedc52c4+10:2407:x1873906077060608:12345-192.168.201.52@tcp:103:ldlm_bl.0 Request processed in 476us (594us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096278.418611:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 59 00010000:00010000:3.0:1787096278.418632:0:21727:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c9a00/0x9eebb2eb7f50d82f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 21727 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.418639:0:21727:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c9a00/0x9eebb2eb7f50d82f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.418647:0:21727:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c9a00/0x9eebb2eb7f50d82f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096278.418660:0:21727:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7982f51c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084459776:0@lo:101:mdt00_005.0 00000100:00100000:3.0:1787096278.418691:0:21727:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096278.418751:0:6482:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084459776 00000100:00100000:0.0:1787096278.418758:0:6482:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084459776 00000100:00100000:3.0:1787096278.418766:0:21727:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7aa07c8600 going to sleep for 11 seconds 00000100:00100000:0.0:1787096278.418800:0:6482:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 150 00000100:00100000:0.0:1787096278.418809:0:6482:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7982f53100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:21727:x1873906084459776:12345-0@lo:101:mdt00_005.0 00010000:00010000:0.0:1787096278.418819:0:6482:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096278.418833:0:6482:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763200/0x9eebb2eb7f50d836 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x9eebb2eb7f50d82f expref: -99 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.418853:0:6482:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763200/0x9eebb2eb7f50d836 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d82f expref: 9 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.418866:0:6482:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763200/0x9eebb2eb7f50d836 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d82f expref: 9 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.418874:0:6482:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9280d55e (ffff8c7a92c71a00) 00010000:00010000:0.0:1787096278.418877:0:6482:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096278.418887:0:6482:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7abd763200, rc 0) 00000100:00100000:0.0:1787096278.418933:0:6482:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7982f53100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:21727:x1873906084459776:12345-0@lo:101:mdt00_005.0 Request processed in 125us (241us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096278.418945:0:6482:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 150 00000100:00100000:3.0:1787096278.418967:0:21727:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7982f51c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084459776:0@lo:101:mdt00_005.0 00010000:00010000:3.0:1787096278.418981:0:21727:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c9a00/0x9eebb2eb7f50d82f lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x9eebb2eb7f50d836 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.418993:0:21727:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c9a00/0x9eebb2eb7f50d82f lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d836 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787096278.419006:0:21727:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1787096278.419035:0:21727:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7982f51c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084459904:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787096278.419067:0:21727:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096278.419086:0:21727:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7aa07c83c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787096278.419109:0:6481:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084459904 00000100:00100000:0.0:1787096278.419116:0:6481:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084459904 00000100:00100000:0.0:1787096278.419147:0:6481:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 151 00000100:00100000:0.0:1787096278.419156:0:6481:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7982f52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:21727:x1873906084459904:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787096278.419221:0:6481:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7982f52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:21727:x1873906084459904:12345-0@lo:1000:mdt00_005.0 Request processed in 65us (152us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096278.419230:0:6481:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 151 00000100:00100000:3.0:1787096278.419233:0:21727:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7982f51c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084459904:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787096278.419273:0:21727:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7982f51c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084460032:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787096278.419304:0:21727:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096278.419319:0:21727:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7aa07c83c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787096278.419338:0:7980:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084460032 00000100:00100000:2.0:1787096278.419343:0:7980:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084460032 00000100:00100000:2.0:1787096278.419403:0:7980:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 152 00000100:00100000:2.0:1787096278.419422:0:7980:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7982f51180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:21727:x1873906084460032:12345-0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1787096278.419482:0:7980:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7982f51180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:21727:x1873906084460032:12345-0@lo:1000:mdt00_005.0 Request processed in 71us (177us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787096278.419488:0:7980:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 152 00000100:00100000:3.0:1787096278.419494:0:21727:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7982f51c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084460032:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787096278.419592:0:21727:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7982f51c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084460160:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787096278.419623:0:21727:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096278.419648:0:21727:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7aa07c83c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787096278.419675:0:7407:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084460160 00000100:00100000:1.0:1787096278.419682:0:7407:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084460160 00000100:00100000:1.0:1787096278.419718:0:7407:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 153 00000100:00100000:1.0:1787096278.419727:0:7407:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7982f53b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:21727:x1873906084460160:12345-0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1787096278.419803:0:7407:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7982f53b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:21727:x1873906084460160:12345-0@lo:1000:mdt00_005.0 Request processed in 76us (178us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787096278.419814:0:7407:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 153 00000100:00100000:3.0:1787096278.419833:0:21727:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7982f51c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084460160:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787096278.419876:0:21727:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7982f51c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084460288:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787096278.419897:0:21727:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096278.419913:0:21727:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7aa07c83c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787096278.419937:0:6482:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084460288 00000100:00100000:0.0:1787096278.419944:0:6482:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084460288 00000100:00100000:0.0:1787096278.419978:0:6482:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 154 00000100:00100000:0.0:1787096278.419986:0:6482:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7982f50700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:21727:x1873906084460288:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787096278.420057:0:6482:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7982f50700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:21727:x1873906084460288:12345-0@lo:1000:mdt00_005.0 Request processed in 70us (156us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096278.420069:0:6482:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 154 00000100:00100000:3.0:1787096278.420077:0:21727:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7982f51c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084460288:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787096278.420116:0:21727:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7982f51c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084460416:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787096278.420136:0:21727:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096278.420151:0:21727:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7aa07c83c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787096278.420163:0:6481:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084460416 00000100:00100000:0.0:1787096278.420169:0:6481:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084460416 00000100:00100000:0.0:1787096278.420200:0:6481:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 155 00000100:00100000:0.0:1787096278.420207:0:6481:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7982f52300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:21727:x1873906084460416:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787096278.420291:0:6481:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7982f52300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:21727:x1873906084460416:12345-0@lo:1000:mdt00_005.0 Request processed in 84us (152us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096278.420302:0:6481:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 155 00000100:00100000:3.0:1787096278.420321:0:21727:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7982f51c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084460416:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787096278.420422:0:21727:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7982f51c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084460544:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787096278.420443:0:21727:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096278.420458:0:21727:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7aa07c83c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787096278.420474:0:7980:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084460544 00000100:00100000:2.0:1787096278.420479:0:7980:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084460544 00000100:00100000:2.0:1787096278.420526:0:7980:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 156 00000100:00100000:2.0:1787096278.420538:0:7980:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7982f52a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:21727:x1873906084460544:12345-0@lo:1000:mdt00_005.0 00000100:00100000:2.0:1787096278.420586:0:7980:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7982f52a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:21727:x1873906084460544:12345-0@lo:1000:mdt00_005.0 Request processed in 49us (141us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096278.420594:0:7980:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 156 00000100:00100000:3.0:1787096278.420613:0:21727:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7982f51c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084460544:0@lo:1000:mdt00_005.0 00000040:00080000:3.0:1787096278.421439:0:21727:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].3, 2152 off38008 00000040:00080000:3.0:1787096278.421529:0:21727:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].3, 2352 off38488 00000004:00080000:3.0:1787096278.421537:0:21727:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff8c7aa07c8600 with version 6 00000004:00080000:1.0:1787096278.421577:0:7405:0:(osp_trans.c:1330:osp_get_next_request()) ou ffff8c7aad23da80 version 6 rpc_version 6 00000020:00100000:1.0:1787096278.421610:0:7405:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787096278.421625:0:7405:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8c7983a6ad80 x1873906084460800/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787096278.421643:0:7405:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7983a6ad80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7405:1873906084460800:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787096278.421678:0:7405:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096278.421705:0:7405:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7abf92c180 going to sleep for 11 seconds 00000100:00100000:3.0:1787096278.421710:0:7407:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084460800 00000100:00100000:3.0:1787096278.421717:0:7407:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084460800 00000100:00100000:3.0:1787096278.421774:0:7407:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 157 00000100:00100000:3.0:1787096278.421783:0:7407:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983a6bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7405:x1873906084460800:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787096278.422298:0:7407:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983a6bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7405:x1873906084460800:12345-0@lo:1000:osp_up0-1.0 Request processed in 516us (618us total) trans 4294967363 rc 0/0 00000100:00100000:3.0:1787096278.422305:0:7407:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 157 00000100:00100000:1.0:1787096278.422370:0:7405:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7983a6ad80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7405:1873906084460800:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787096278.422466:0:21727:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c8c00/0x9eebb2eb7f50d83d lrc: 3/0,1 mode: --/PW res: [0x240000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 21727 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.422476:0:21727:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c8c00/0x9eebb2eb7f50d83d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.422486:0:21727:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c8c00/0x9eebb2eb7f50d83d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.422516:0:21727:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7aa2750200/0x9eebb2eb7f50d844 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 21727 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.422525:0:21727:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff8c7aa2750200/0x9eebb2eb7f50d844 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.422534:0:21727:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7aa2750200/0x9eebb2eb7f50d844 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.422568:0:21727:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7aa2750000/0x9eebb2eb7f50d84b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 21727 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.422576:0:21727:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7aa2750000/0x9eebb2eb7f50d84b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.422585:0:21727:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7aa2750000/0x9eebb2eb7f50d84b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096278.422601:0:21727:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7982f51c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084460928:0@lo:101:mdt00_005.0 00000100:00100000:3.0:1787096278.422631:0:21727:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096278.422649:0:21727:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7ab3f50300 going to sleep for 11 seconds 00000100:00100000:0.0:1787096278.422690:0:6482:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084460928 00000100:00100000:0.0:1787096278.422698:0:6482:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084460928 00000100:00100000:0.0:1787096278.422740:0:6482:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 158 00000100:00100000:0.0:1787096278.422759:0:6482:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7982f50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:21727:x1873906084460928:12345-0@lo:101:mdt00_005.0 00010000:00010000:0.0:1787096278.422773:0:6482:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096278.422789:0:6482:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763400/0x9eebb2eb7f50d852 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x9eebb2eb7f50d84b expref: -99 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.422809:0:6482:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763400/0x9eebb2eb7f50d852 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d84b expref: 11 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.422821:0:6482:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763400/0x9eebb2eb7f50d852 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d84b expref: 11 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.422829:0:6482:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff8c7a92c71400) 00010000:00010000:0.0:1787096278.422832:0:6482:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096278.422834:0:6482:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7abd763400, rc 0) 00000100:00100000:0.0:1787096278.422884:0:6482:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7982f50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:21727:x1873906084460928:12345-0@lo:101:mdt00_005.0 Request processed in 127us (250us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096278.422895:0:6482:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 158 00000100:00100000:3.0:1787096278.422919:0:21727:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7982f51c00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:21727:1873906084460928:0@lo:101:mdt00_005.0 00010000:00010000:3.0:1787096278.422937:0:21727:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7aa2750000/0x9eebb2eb7f50d84b lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x9eebb2eb7f50d852 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.422948:0:21727:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7aa2750000/0x9eebb2eb7f50d84b lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d852 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787096278.422965:0:21727:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000005441223a reply state 000000005ce31d23 transno 4294967305 ns: mdt-lustre-MDT0001_UUID lock: ffff8c7aa2750200/0x9eebb2eb7f50d844 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.422978:0:21727:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff8c7aa2750200/0x9eebb2eb7f50d844 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.422986:0:21727:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffff8c7aadca5600) 00010000:00010000:3.0:1787096278.422989:0:21727:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787096278.422993:0:21727:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787096278.423001:0:21727:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7aa2750000/0x9eebb2eb7f50d84b lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x9eebb2eb7f50d852 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.423010:0:21727:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7aa2750000/0x9eebb2eb7f50d84b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x9eebb2eb7f50d852 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787096278.423021:0:21727:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000401:0x5:0x0] 00000004:00010000:3.0:1787096278.423027:0:21727:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000005441223a reply state 000000005ce31d23 transno 4294967305 ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c8c00/0x9eebb2eb7f50d83d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.423037:0:21727:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c8c00/0x9eebb2eb7f50d83d lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.423052:0:21727:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x2:0x0].0x0 (ffff8c7aadca5400) 00010000:00010000:3.0:1787096278.423054:0:21727:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787096278.423057:0:21727:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:3.0:1787096278.423063:0:21727:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x2:0x0]: returning size 4096 00000004:00080000:3.0:1787096278.423070:0:21727:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787096278.423079:0:21727:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.423087:0:21727:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.423095:0:21727:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.423119:0:21727:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787096278.423125:0:21727:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:3.0:1787096278.423130:0:21727:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c9a00/0x9eebb2eb7f50d82f lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x9eebb2eb7f50d836 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.423139:0:21727:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c9a00/0x9eebb2eb7f50d82f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x9eebb2eb7f50d836 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.423157:0:6465:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787096278.423166:0:6465:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.423177:0:6465:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7abd762000 already unused, calling callback (ffffffffc1765ee0) 00010000:00010000:0.0:1787096278.423183:0:6465:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787096278.423192:0:6465:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096278.423227:0:21727:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983923b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:f9858421-28a9-4744-99eb-72e5bedc52c4+14:34319:x1873906077060480:12345-192.168.201.52@tcp:36:lfs.0 Request processed in 7529us (7682us total) trans 4294967305 rc 0/0 00000100:00100000:3.0:1787096278.423235:0:21727:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 165 00010000:00010000:0.0:1787096278.423241:0:6465:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.423251:0:6465:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787096278.423285:0:6465:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096278.423288:0:3626:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7aadf2d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3626:1873906084461056:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787096278.423310:0:6465:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7abd762000/0x9eebb2eb7f50d821 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x9eebb2eb7f50d828 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.423336:0:6465:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787096278.423338:0:3626:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787096278.423341:0:6465:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096278.423344:0:6465:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787096278.423440:0:10225:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084461056 00000100:00100000:2.0:1787096278.423445:0:10225:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084461056 00000100:00100000:2.0:1787096278.423486:0:10225:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8 00000100:00100000:2.0:1787096278.423492:0:10225:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7982f51c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+11:3626:x1873906084461056:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787096278.423512:0:10225:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787096278.423523:0:10225:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7983881800/0x9eebb2eb7f50d828 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x9eebb2eb7f50d821 expref: 11 pid: 7407 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.423535:0:10225:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8c7a92c71b00) 00010000:00010000:2.0:1787096278.423541:0:10225:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787096278.423573:0:10225:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7982f51c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+10:3626:x1873906084461056:12345-0@lo:103:ldlm_bl.0 Request processed in 81us (236us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096278.423578:0:10225:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8 00000100:00100000:3.0:1787096278.423612:0:3626:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7aadf2d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3626:1873906084461056:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787096278.435329:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:0.0:1787096278.435389:0:8384:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077060736 00000100:00100000:0.0:1787096278.435397:0:8384:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077060736 00000100:00100000:0.0:1787096278.435439:0:8384:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 166 00000100:00100000:0.0:1787096278.435453:0:8384:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7a883f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f9858421-28a9-4744-99eb-72e5bedc52c4+10:34322:x1873906077060736:12345-192.168.201.52@tcp:101:lfs.0 00010000:00010000:0.0:1787096278.435469:0:8384:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096278.435488:0:8384:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd762000/0x9eebb2eb7f50d859 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x4ef7bc65b7b41107 expref: -99 pid: 8384 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.435566:0:8384:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763800/0x9eebb2eb7f50d860 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.435580:0:8384:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763800/0x9eebb2eb7f50d860 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.435591:0:8384:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763800/0x9eebb2eb7f50d860 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787096278.435642:0:8384:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:0.0:1787096278.435647:0:8384:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787096278.435668:0:8384:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763800/0x9eebb2eb7f50d860 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.435695:0:8384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd762000/0x9eebb2eb7f50d859 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41107 expref: 12 pid: 8384 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.435720:0:8384:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763800/0x9eebb2eb7f50d860 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41107 expref: 11 pid: 8384 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.435729:0:8384:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8c7a92c71300) 00010000:00010000:0.0:1787096278.435733:0:8384:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787096278.435737:0:8384:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7abd763800, rc 0) 00000100:00100000:0.0:1787096278.435801:0:8384:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7a883f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f9858421-28a9-4744-99eb-72e5bedc52c4+11:34322:x1873906077060736:12345-192.168.201.52@tcp:101:lfs.0 Request processed in 353us (477us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096278.435810:0:8384:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 166 00000100:00100000:1.0:1787096278.437254:0:3621:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:2.0:1787096278.437312:0:10275:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077060864 00000100:00100000:2.0:1787096278.437318:0:10275:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077060864 00000100:00100000:2.0:1787096278.437351:0:10275:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 167 00000100:00100000:2.0:1787096278.437390:0:10275:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983a68700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f9858421-28a9-4744-99eb-72e5bedc52c4+11:34322:x1873906077060864:12345-192.168.201.52@tcp:101:lfs.0 00010000:00010000:2.0:1787096278.437402:0:10275:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787096278.437413:0:10275:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813eda00/0x9eebb2eb7f50d867 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x4ef7bc65b7b4110e expref: -99 pid: 10275 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.437449:0:10275:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ec400/0x9eebb2eb7f50d86e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.437463:0:10275:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ec400/0x9eebb2eb7f50d86e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.437469:0:10275:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ec400/0x9eebb2eb7f50d86e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.437477:0:10275:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d875 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.437484:0:10275:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763200/0x9eebb2eb7f50d836 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d82f expref: 9 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.437490:0:10275:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d875 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.437529:0:10275:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763200/0x9eebb2eb7f50d836 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x9eebb2eb7f50d82f expref: 9 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096278.437542:0:10275:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7a883f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:10275:1873906084461184:0@lo:104: 00000100:00100000:2.0:1787096278.437571:0:10275:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096278.437614:0:6460:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084461184 00000100:00100000:2.0:1787096278.437620:0:10275:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c79831ab480 going to sleep for 11 seconds 00000100:00100000:3.0:1787096278.437625:0:6460:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084461184 00000100:00100000:3.0:1787096278.437671:0:6460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 39 00000100:00100000:3.0:1787096278.437681:0:6460:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7a883f7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:10275:x1873906084461184:12345-0@lo:104: 00010000:00010000:3.0:1787096278.437697:0:6460:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c9a00/0x9eebb2eb7f50d82f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x9eebb2eb7f50d836 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.437774:0:6460:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787096278.437778:0:6460:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7a883f7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:10275:x1873906084461184:12345-0@lo:104: Request processed in 101us (206us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096278.437785:0:6460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 39 00010000:00010000:0.0:1787096278.437814:0:6466:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787096278.437822:0:6466:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c9a00/0x9eebb2eb7f50d82f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x9eebb2eb7f50d836 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096278.437829:0:10275:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7a883f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:10275:1873906084461184:0@lo:104: 00010000:00010000:0.0:1787096278.437836:0:6466:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7a869c9a00 already unused, calling callback (ffffffffc1765ee0) 00010000:00010000:0.0:1787096278.437850:0:6466:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c9a00/0x9eebb2eb7f50d82f lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x9eebb2eb7f50d836 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.437862:0:10275:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d875 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787096278.437864:0:6466:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c9a00/0x9eebb2eb7f50d82f lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x9eebb2eb7f50d836 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.437929:0:6466:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c9a00/0x9eebb2eb7f50d82f lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x9eebb2eb7f50d836 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.437935:0:6466:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787096278.437960:0:6466:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c9a00/0x9eebb2eb7f50d82f lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x9eebb2eb7f50d836 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.437966:0:6466:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a869c9a00/0x9eebb2eb7f50d82f lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x9eebb2eb7f50d836 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096278.437981:0:3626:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:0.0:1787096278.437989:0:6466:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096278.437996:0:6466:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096278.438001:0:6466:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787096278.438003:0:3626:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7aadf2df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3626:1873906084461312:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787096278.438035:0:3626:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096278.438069:0:7401:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084461312 00000100:00100000:0.0:1787096278.438074:0:7401:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084461312 00010000:00010000:0.0:1787096278.438092:0:7401:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763200/0x9eebb2eb7f50d836 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x9eebb2eb7f50d82f expref: 10 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096278.438119:0:7401:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 60 00000100:00100000:0.0:1787096278.438126:0:7401:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7a90587050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+10:3626:x1873906084461312:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787096278.438137:0:7401:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:2.0:1787096278.439430:0:6456:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967363 is committed 00000001:00080000:2.0:1787096278.439456:0:6456:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967361 is committed 00010000:00010000:0.0:1787096278.439539:0:7401:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763200/0x9eebb2eb7f50d836 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x9eebb2eb7f50d82f expref: 8 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.439541:0:3613:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750400/0x9eebb2eb7f50d71e lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.439566:0:3613:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750400/0x9eebb2eb7f50d71e lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.439568:0:7401:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9280d55e (ffff8c7a92c71a00) 00010000:00010000:1.0:1787096278.439586:0:3613:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750400/0x9eebb2eb7f50d71e lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.439586:0:7401:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d875 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.439626:0:3613:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7aa2750400 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:1.0:1787096278.439629:0:3613:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750400/0x9eebb2eb7f50d71e lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.439661:0:3613:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750400/0x9eebb2eb7f50d71e lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.439667:0:7401:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d875 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.439673:0:3613:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2:0x0].0x7b711ae6 (ffff8c7a836ece00) 00010000:00010000:1.0:1787096278.439678:0:3613:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096278.439678:0:7401:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d875 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.439683:0:3613:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750400/0x9eebb2eb7f50d71e lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.439690:0:3613:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750400/0x9eebb2eb7f50d71e lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.439732:0:7401:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787096278.439736:0:3613:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d7a3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.439743:0:3613:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d7a3 lrc: 2/0,0 mode: TXN/TXN res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.439749:0:3613:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d7a3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.439754:0:3613:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7a89766c00 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:1.0:1787096278.439757:0:3613:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d7a3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.439762:0:3613:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d7a3 lrc: 4/0,0 mode: TXN/TXN res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.439763:0:10275:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d875 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.439774:0:3613:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2:0x0].0x7b711ae6 (ffff8c7a836ece00) 00000100:00100000:0.0:1787096278.439776:0:7401:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7a90587050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+7:3626:x1873906084461312:12345-0@lo:103:ldlm_bl.0 Request processed in 1647us (1736us total) trans 0 rc 0/0 00010000:00010000:1.0:1787096278.439786:0:3613:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787096278.439791:0:3613:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d7a3 lrc: 2/0,0 mode: --/TXN res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096278.439798:0:7401:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 60 00000100:00100000:3.0:1787096278.439810:0:3626:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7aadf2df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3626:1873906084461312:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787096278.439823:0:10275:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d875 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.439834:0:3613:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d7a3 lrc: 0/0,0 mode: --/TXN res: [0x200000401:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096278.439915:0:10275:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7a883f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:10275:1873906084461440:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787096278.439955:0:10275:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096278.439977:0:10275:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c79831abc00 going to sleep for 11 seconds 00000100:00100000:0.0:1787096278.439995:0:6481:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084461440 00000100:00100000:0.0:1787096278.440001:0:6481:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084461440 00000100:00100000:0.0:1787096278.440040:0:6481:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 159 00000100:00100000:0.0:1787096278.440050:0:6481:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7a883f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:10275:x1873906084461440:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787096278.440119:0:6481:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7a883f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:10275:x1873906084461440:12345-0@lo:1000:mdt00_004.0 Request processed in 72us (165us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096278.440127:0:6481:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 159 00000100:00100000:2.0:1787096278.440163:0:10275:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7a883f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:10275:1873906084461440:0@lo:1000:mdt00_004.0 00010000:00010000:2.0:1787096278.440191:0:10275:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ec600/0x9eebb2eb7f50d87c lrc: 3/1,0 mode: --/PR res: [0x240000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440199:0:10275:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ec600/0x9eebb2eb7f50d87c lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x2:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440205:0:10275:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ec600/0x9eebb2eb7f50d87c lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x2:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787096278.440216:0:10275:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ec600/0x9eebb2eb7f50d87c lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x2:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440224:0:10275:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ec400/0x9eebb2eb7f50d86e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440228:0:10275:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ec400/0x9eebb2eb7f50d86e lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440233:0:10275:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ec400/0x9eebb2eb7f50d86e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440236:0:10275:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7a813ec400 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:2.0:1787096278.440238:0:10275:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ec400/0x9eebb2eb7f50d86e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440247:0:10275:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ec400/0x9eebb2eb7f50d86e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440253:0:10275:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8c7a92c71300) 00010000:00010000:2.0:1787096278.440255:0:10275:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096278.440258:0:10275:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ec400/0x9eebb2eb7f50d86e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440262:0:10275:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ec400/0x9eebb2eb7f50d86e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440269:0:10275:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d875 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440273:0:10275:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d875 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440277:0:10275:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d875 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440282:0:10275:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7a813ed200 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:2.0:1787096278.440285:0:10275:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d875 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440291:0:10275:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d875 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440299:0:10275:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9280d55e (ffff8c7a92c71a00) 00010000:00010000:2.0:1787096278.440301:0:10275:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096278.440304:0:10275:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d875 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440321:0:10275:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d875 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440351:0:10275:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813eda00/0x9eebb2eb7f50d867 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b4110e expref: 8 pid: 10275 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440395:0:10275:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7a813ec600/0x9eebb2eb7f50d87c lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x2:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b4110e expref: 7 pid: 10275 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.440401:0:10275:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x2:0x0].0x0 (ffff8c7aadca8700) 00010000:00010000:2.0:1787096278.440403:0:10275:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787096278.440407:0:10275:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7a813ec600, rc 0) 00000100:00100000:2.0:1787096278.440453:0:10275:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983a68700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f9858421-28a9-4744-99eb-72e5bedc52c4+7:34322:x1873906077060864:12345-192.168.201.52@tcp:101:lfs.0 Request processed in 3066us (3202us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096278.440461:0:10275:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 167 00000100:00100000:2.0:1787096278.441481:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:0.0:1787096278.441525:0:6478:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077060992 00000100:00100000:0.0:1787096278.441530:0:6478:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077060992 00000100:00100000:0.0:1787096278.441563:0:6478:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 168 00000100:00100000:0.0:1787096278.441569:0:6478:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7a883f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f9858421-28a9-4744-99eb-72e5bedc52c4+14:34322:x1873906077060992:12345-192.168.201.52@tcp:101:lfs.0 00010000:00010000:0.0:1787096278.441581:0:6478:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096278.441593:0:6478:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd763c00/0x9eebb2eb7f50d883 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x4ef7bc65b7b41115 expref: -99 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.441651:0:6478:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762600/0x9eebb2eb7f50d88a lrc: 3/1,0 mode: --/PR res: [0x240000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.441676:0:6478:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762600/0x9eebb2eb7f50d88a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x2:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.441684:0:6478:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762600/0x9eebb2eb7f50d88a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x2:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6478 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787096278.441757:0:6478:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x2:0x0]: returning size 4096 00000004:00200000:0.0:1787096278.441762:0:6478:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787096278.441788:0:6478:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd763c00/0x9eebb2eb7f50d883 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x2:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41115 expref: 16 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.441798:0:6478:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7abd762600/0x9eebb2eb7f50d88a lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x2:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41115 expref: 15 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.441804:0:6478:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x2:0x0].0x0 (ffff8c7aadca5400) 00010000:00010000:0.0:1787096278.441806:0:6478:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787096278.441808:0:6478:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7abd762600, rc 0) 00000100:00100000:0.0:1787096278.441846:0:6478:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7a883f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f9858421-28a9-4744-99eb-72e5bedc52c4+15:34322:x1873906077060992:12345-192.168.201.52@tcp:101:lfs.0 Request processed in 277us (366us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096278.441850:0:6478:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 168 00000100:00100000:0.0:1787096278.442747:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:2.0:1787096278.442784:0:6477:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077061120 00000100:00100000:2.0:1787096278.442790:0:6477:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077061120 00000100:00100000:2.0:1787096278.442834:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 169 00000100:00100000:2.0:1787096278.442850:0:6477:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7aadf2df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f9858421-28a9-4744-99eb-72e5bedc52c4+15:34322:x1873906077061120:12345-192.168.201.52@tcp:101:lfs.0 00010000:00010000:2.0:1787096278.442864:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787096278.442877:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a813eca00/0x9eebb2eb7f50d891 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x4ef7bc65b7b4111c expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.442916:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a813ed800/0x9eebb2eb7f50d898 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.442940:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a813ed800/0x9eebb2eb7f50d898 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.442950:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a813ed800/0x9eebb2eb7f50d898 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787096278.442992:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000400:0x5:0x0]: returning size 4096 00000004:00200000:2.0:1787096278.442997:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787096278.443024:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a813eca00/0x9eebb2eb7f50d891 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b4111c expref: 17 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.443040:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a813ed800/0x9eebb2eb7f50d898 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b4111c expref: 16 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.443049:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffff8c7aadca5600) 00010000:00010000:2.0:1787096278.443052:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787096278.443056:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7a813ed800, rc 0) 00000100:00100000:2.0:1787096278.443099:0:6477:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7aadf2df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f9858421-28a9-4744-99eb-72e5bedc52c4+16:34322:x1873906077061120:12345-192.168.201.52@tcp:101:lfs.0 Request processed in 256us (351us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096278.443107:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 169 00000100:00100000:1.0:1787096278.443909:0:3621:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:3.0:1787096278.443959:0:6476:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077061248 00000100:00100000:3.0:1787096278.443964:0:6476:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077061248 00000100:00100000:3.0:1787096278.443993:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 170 00000100:00100000:3.0:1787096278.444002:0:6476:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983a6a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f9858421-28a9-4744-99eb-72e5bedc52c4+7:34322:x1873906077061248:12345-192.168.201.52@tcp:101:lfs.0 00010000:00010000:3.0:1787096278.444020:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787096278.444032:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2751400/0x9eebb2eb7f50d89f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x4ef7bc65b7b41123 expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.444073:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750a00/0x9eebb2eb7f50d8a6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.444082:0:6476:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763400/0x9eebb2eb7f50d852 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d84b expref: 6 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.444095:0:6476:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750a00/0x9eebb2eb7f50d8a6 lrc: 4/1,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.444113:0:6476:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763400/0x9eebb2eb7f50d852 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x9eebb2eb7f50d84b expref: 6 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096278.444122:0:6476:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7982f53800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1873906084461568:0@lo:104: 00000100:00100000:3.0:1787096278.444146:0:6476:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096278.444186:0:6476:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c7ab3f50240 going to sleep for 11 seconds 00000100:00100000:1.0:1787096278.444188:0:7399:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084461568 00000100:00100000:1.0:1787096278.444193:0:7399:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084461568 00000100:00100000:1.0:1787096278.444225:0:7399:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 40 00000100:00100000:1.0:1787096278.444231:0:7399:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7982f51c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6476:x1873906084461568:12345-0@lo:104: 00010000:00010000:1.0:1787096278.444241:0:7399:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7aa2750000/0x9eebb2eb7f50d84b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x9eebb2eb7f50d852 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096278.444291:0:7399:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787096278.444297:0:7399:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7982f51c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6476:x1873906084461568:12345-0@lo:104: Request processed in 66us (149us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096278.444304:0:7399:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 40 00010000:00010000:2.0:1787096278.444317:0:7400:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:3.0:1787096278.444318:0:6476:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7982f53800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1873906084461568:0@lo:104: 00010000:00010000:2.0:1787096278.444326:0:7400:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7aa2750000/0x9eebb2eb7f50d84b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x9eebb2eb7f50d852 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.444333:0:6476:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750a00/0x9eebb2eb7f50d8a6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.444334:0:7400:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7aa2750000 already unused, calling callback (ffffffffc1765ee0) 00010000:00010000:2.0:1787096278.444342:0:7400:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7aa2750000/0x9eebb2eb7f50d84b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x9eebb2eb7f50d852 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787096278.444381:0:7400:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7aa2750000/0x9eebb2eb7f50d84b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x9eebb2eb7f50d852 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.444407:0:7400:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7aa2750000/0x9eebb2eb7f50d84b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x9eebb2eb7f50d852 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.444412:0:7400:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787096278.444435:0:7400:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7aa2750000/0x9eebb2eb7f50d84b lrc: 1/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x9eebb2eb7f50d852 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.444441:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7aa2750000/0x9eebb2eb7f50d84b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x9eebb2eb7f50d852 expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096278.444448:0:3628:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7a883f5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3628:1873906084461696:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787096278.444462:0:7400:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787096278.444466:0:7400:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787096278.444468:0:7400:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787096278.444488:0:3628:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096278.444539:0:8375:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084461696 00000100:00100000:2.0:1787096278.444546:0:8375:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084461696 00010000:00010000:2.0:1787096278.444569:0:8375:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763400/0x9eebb2eb7f50d852 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x9eebb2eb7f50d84b expref: 7 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096278.444602:0:8375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 61 00000100:00100000:2.0:1787096278.444611:0:8375:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7aadf2d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+7:3628:x1873906084461696:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787096278.444622:0:8375:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787096278.444935:0:8375:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763400/0x9eebb2eb7f50d852 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x9eebb2eb7f50d84b expref: 7 pid: 6482 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.444949:0:8375:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff8c7a92c71400) 00010000:00010000:2.0:1787096278.444960:0:8375:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750a00/0x9eebb2eb7f50d8a6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.444968:0:8375:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750a00/0x9eebb2eb7f50d8a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.444976:0:8375:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750a00/0x9eebb2eb7f50d8a6 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.445000:0:8375:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787096278.445028:0:6476:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750a00/0x9eebb2eb7f50d8a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.445039:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750a00/0x9eebb2eb7f50d8a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096278.445048:0:8375:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7aadf2d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+6:3628:x1873906084461696:12345-0@lo:103:ldlm_bl.0 Request processed in 438us (558us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096278.445057:0:8375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 61 00000004:00200000:3.0:1787096278.445065:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000401:0x5:0x0]: returning size 4096 00000004:00200000:3.0:1787096278.445069:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787096278.445090:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2751400/0x9eebb2eb7f50d89f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41123 expref: 9 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.445108:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7aa2750a00/0x9eebb2eb7f50d8a6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41123 expref: 8 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.445115:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff8c7a92c71400) 00000100:00100000:0.0:1787096278.445120:0:3628:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7a883f5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3628:1873906084461696:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787096278.445131:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787096278.445135:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7aa2750a00, rc 0) 00000100:00100000:3.0:1787096278.445168:0:6476:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983a6a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f9858421-28a9-4744-99eb-72e5bedc52c4+8:34322:x1873906077061248:12345-192.168.201.52@tcp:101:lfs.0 Request processed in 1168us (1259us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096278.445175:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 170 00000100:00100000:0.0:1787096278.446189:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:3.0:1787096278.446244:0:21727:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077061376 00000100:00100000:3.0:1787096278.446258:0:21727:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077061376 00000100:00100000:3.0:1787096278.446289:0:21727:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 171 00000100:00100000:3.0:1787096278.446297:0:21727:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7a879f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:f9858421-28a9-4744-99eb-72e5bedc52c4+16:34322:x1873906077061376:12345-192.168.201.52@tcp:101:lfs.0 00010000:00010000:3.0:1787096278.446306:0:21727:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787096278.446314:0:21727:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7aa2751c00/0x9eebb2eb7f50d8ad lrc: 2/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x4ef7bc65b7b4112a expref: -99 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446350:0:21727:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9e00/0x9eebb2eb7f50d8b4 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 21727 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446389:0:21727:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9e00/0x9eebb2eb7f50d8b4 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446400:0:21727:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9e00/0x9eebb2eb7f50d8b4 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446445:0:21727:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d8bb lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 21727 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446454:0:21727:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d8bb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446459:0:21727:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d8bb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446520:0:21727:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9e00/0x9eebb2eb7f50d8b4 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446526:0:21727:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9e00/0x9eebb2eb7f50d8b4 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446530:0:21727:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9e00/0x9eebb2eb7f50d8b4 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446534:0:21727:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9e00/0x9eebb2eb7f50d8b4 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446537:0:21727:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7a869c9e00 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:3.0:1787096278.446539:0:21727:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9e00/0x9eebb2eb7f50d8b4 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446544:0:21727:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9e00/0x9eebb2eb7f50d8b4 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446550:0:21727:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffff8c7aadca5600) 00010000:00010000:3.0:1787096278.446552:0:21727:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787096278.446555:0:21727:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9e00/0x9eebb2eb7f50d8b4 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446559:0:21727:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9e00/0x9eebb2eb7f50d8b4 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446566:0:21727:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d8bb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446569:0:21727:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d8bb lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446578:0:21727:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d8bb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446582:0:21727:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7a869c9800 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:3.0:1787096278.446584:0:21727:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d8bb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446588:0:21727:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d8bb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446592:0:21727:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0xae2548a7 (ffff8c7aadca5d00) 00010000:00010000:3.0:1787096278.446594:0:21727:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787096278.446596:0:21727:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d8bb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446599:0:21727:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a869c9800/0x9eebb2eb7f50d8bb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 21727 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446616:0:21727:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7aa2751c00/0x9eebb2eb7f50d8ad lrc: 1/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b4112a expref: 17 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446622:0:21727:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffff8c7aadca5600) 00010000:00010000:3.0:1787096278.446624:0:21727:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787096278.446626:0:21727:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7aa2751c00/0x9eebb2eb7f50d8ad lrc: 0/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b4112a expref: 17 pid: 21727 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096278.446632:0:21727:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7aa2751c00, rc 0) 00000100:00100000:3.0:1787096278.446662:0:21727:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7a879f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:f9858421-28a9-4744-99eb-72e5bedc52c4+16:34322:x1873906077061376:12345-192.168.201.52@tcp:101:lfs.0 Request processed in 367us (477us total) trans 0 rc 301/301 00000100:00100000:3.0:1787096278.446674:0:21727:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 171 00000100:00100000:0.0:1787096278.447873:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:2.0:1787096278.447958:0:8384:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077061632 00000100:00100000:2.0:1787096278.447964:0:8384:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077061632 00000100:00100000:2.0:1787096278.448003:0:8384:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 172 00000100:00100000:2.0:1787096278.448011:0:8384:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983921500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f9858421-28a9-4744-99eb-72e5bedc52c4+16:34322:x1873906077061632:12345-192.168.201.52@tcp:101:lfs.0 00010000:00010000:2.0:1787096278.448021:0:8384:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787096278.448023:0:8384:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787096278.448036:0:8384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a813ed800/0x9eebb2eb7f50d898 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b4111c expref: 16 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.448049:0:8384:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffff8c7aadca5600) 00010000:00010000:2.0:1787096278.448051:0:8384:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787096278.448058:0:8384:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d8c2 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x4ef7bc65b7b41131 expref: -99 pid: 8384 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.448128:0:8384:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d8c9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.448144:0:8384:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d8c9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x0 pid: 8384 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.448152:0:8384:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d8c9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 8384 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096278.448168:0:8384:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7a883f5880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8384:1873906084461824:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787096278.448197:0:8384:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096278.448223:0:8384:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c79831ab480 going to sleep for 11 seconds 00000100:00100000:0.0:1787096278.448226:0:7980:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084461824 00000100:00100000:0.0:1787096278.448237:0:7980:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084461824 00000100:00100000:0.0:1787096278.448267:0:7980:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 160 00000100:00100000:0.0:1787096278.448272:0:7980:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7abd60d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:8384:x1873906084461824:12345-0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787096278.448281:0:7980:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096278.448291:0:7980:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763200/0x9eebb2eb7f50d8d0 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x9eebb2eb7f50d8c9 expref: -99 pid: 7980 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.448310:0:7980:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763200/0x9eebb2eb7f50d8d0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d8c9 expref: 7 pid: 7980 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.448323:0:7980:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8c7abd763200/0x9eebb2eb7f50d8d0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x9eebb2eb7f50d8c9 expref: 7 pid: 7980 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.448332:0:7980:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8c7a92c71300) 00010000:00010000:0.0:1787096278.448335:0:7980:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1787096278.448338:0:7980:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7abd763200, rc 0) 00000100:00100000:0.0:1787096278.448429:0:7980:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7abd60d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:8384:x1873906084461824:12345-0@lo:101:mdt00_003.0 Request processed in 154us (227us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096278.448437:0:7980:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 160 00000100:00100000:2.0:1787096278.448469:0:8384:0:(client.c:3102:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967363 gen 2 00000004:00080000:2.0:1787096278.448475:0:8384:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294967362 committed transno 4294967363 00000004:00080000:2.0:1787096278.448517:0:8384:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294967363 committed transno 4294967363 00000100:00100000:2.0:1787096278.448537:0:8384:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7a883f5880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8384:1873906084461824:0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787096278.448549:0:8384:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d8c9 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x9eebb2eb7f50d8d0 expref: -99 pid: 8384 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.448558:0:8384:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d8c9 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x9eebb2eb7f50d8d0 expref: -99 pid: 8384 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787096278.448569:0:8384:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1787096278.448574:0:8384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d8c9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x9eebb2eb7f50d8d0 expref: -99 pid: 8384 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.448580:0:8384:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff8c7a813ec400/0x9eebb2eb7f50d8c9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x9eebb2eb7f50d8d0 expref: -99 pid: 8384 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.448614:0:8384:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d8d7 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.448629:0:8384:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d8d7 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8384 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.448634:0:8384:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d8d7 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8384 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.448645:0:8384:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89767000/0x9eebb2eb7f50d8de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.448650:0:8384:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89767000/0x9eebb2eb7f50d8de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8384 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.448654:0:8384:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89767000/0x9eebb2eb7f50d8de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8384 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096278.448752:0:8384:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7abd60c700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8384:1873906084461952:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787096278.448774:0:8384:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096278.448791:0:8384:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c79831abc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787096278.448810:0:7407:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906084461952 00000100:00100000:3.0:1787096278.448816:0:7407:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906084461952 00000100:00100000:3.0:1787096278.448844:0:7407:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 161 00000100:00100000:3.0:1787096278.448850:0:7407:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7abd60f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:8384:x1873906084461952:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787096278.448938:0:7407:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7abd60f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:8384:x1873906084461952:12345-0@lo:1000:mdt00_003.0 Request processed in 87us (162us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096278.448944:0:7407:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 161 00000100:00100000:2.0:1787096278.448971:0:8384:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7abd60c700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8384:1873906084461952:0@lo:1000:mdt00_003.0 00010000:00010000:2.0:1787096278.449279:0:8384:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89767c00/0x9eebb2eb7f50d8e5 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8384 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.449287:0:8384:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89767c00/0x9eebb2eb7f50d8e5 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8384 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.449292:0:8384:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89767c00/0x9eebb2eb7f50d8e5 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8384 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787096278.449300:0:8384:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x3:0x0]: returning size 0 00010000:00010000:2.0:1787096278.449319:0:8384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d8d7 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8384 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.449329:0:8384:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d8d7 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8384 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.449333:0:8384:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d8d7 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8384 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.449337:0:8384:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7a89766c00 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:2.0:1787096278.449340:0:8384:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d8d7 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8384 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.449344:0:8384:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d8d7 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8384 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.449350:0:8384:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffff8c7aadca5600) 00010000:00010000:2.0:1787096278.449352:0:8384:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096278.449389:0:8384:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d8d7 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8384 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.449395:0:8384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766c00/0x9eebb2eb7f50d8d7 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8384 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787096278.449409:0:8384:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000505c686c reply state 0000000036a2d704 transno 4294967306 ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89767000/0x9eebb2eb7f50d8de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8384 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.449417:0:8384:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89767000/0x9eebb2eb7f50d8de lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8384 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.449422:0:8384:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0xae2548a7 (ffff8c7aadca8e00) 00010000:00010000:2.0:1787096278.449424:0:8384:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787096278.449426:0:8384:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787096278.449446:0:8384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a813ed200/0x9eebb2eb7f50d8c2 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41131 expref: 18 pid: 8384 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.449460:0:8384:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89767c00/0x9eebb2eb7f50d8e5 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41131 expref: 17 pid: 8384 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.449465:0:8384:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3:0x0].0x0 (ffff8c7aadca8100) 00010000:00010000:2.0:1787096278.449467:0:8384:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787096278.449468:0:8384:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7a89767c00, rc 0) 00000100:00100000:2.0:1787096278.449517:0:8384:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983921500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f9858421-28a9-4744-99eb-72e5bedc52c4+18:34322:x1873906077061632:12345-192.168.201.52@tcp:101:lfs.0 Request processed in 1506us (1645us total) trans 4294967306 rc 0/0 00000100:00100000:2.0:1787096278.449523:0:8384:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 172 00000100:00100000:1.0:1787096278.450710:0:3621:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:2.0:1787096278.450758:0:10275:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077061888 00000100:00100000:2.0:1787096278.450764:0:10275:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077061888 00000100:00100000:2.0:1787096278.450798:0:10275:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 173 00000100:00100000:2.0:1787096278.450806:0:10275:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983a6aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f9858421-28a9-4744-99eb-72e5bedc52c4+18:34322:x1873906077061888:12345-192.168.201.52@tcp:101:lfs.0 00010000:00010000:2.0:1787096278.450817:0:10275:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787096278.450824:0:10275:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766000/0x9eebb2eb7f50d8f3 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x4ef7bc65b7b41138 expref: -99 pid: 10275 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.450868:0:10275:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d8fa lrc: 3/0,1 mode: --/EX res: [0x240000402:0x3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 10275 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.450877:0:10275:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89767c00/0x9eebb2eb7f50d8e5 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41131 expref: 19 pid: 8384 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.450883:0:10275:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d8fa lrc: 4/0,1 mode: --/EX res: [0x240000402:0x3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 10275 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.450900:0:10275:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89767c00/0x9eebb2eb7f50d8e5 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41131 expref: 19 pid: 8384 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.450918:0:10275:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89767c00/0x9eebb2eb7f50d8e5 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41131 expref: 19 pid: 8384 timeout: 328 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096278.450926:0:10275:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8c7abd60c700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:10275:1873906084462080:192.168.201.52@tcp:104: 00000100:00100000:2.0:1787096278.450971:0:10275:0:(client.c:2692:ptlrpc_set_wait()) set ffff8c79831ab180 going to sleep for 11 seconds 00000100:00100000:2.0:1787096278.451749:0:10275:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff8c7abd60c700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:10275:1873906084462080:192.168.201.52@tcp:104: 00010000:00010000:2.0:1787096278.451759:0:10275:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d8fa lrc: 3/0,1 mode: --/EX res: [0x240000402:0x3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10275 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096278.452167:0:3621:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:0.0:1787096278.452212:0:6464:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077062016 00000100:00100000:0.0:1787096278.452216:0:6464:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077062016 00010000:00010000:0.0:1787096278.452234:0:6464:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89767c00/0x9eebb2eb7f50d8e5 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41131 expref: 20 pid: 8384 timeout: 328 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096278.452262:0:6464:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 62 00000100:00100000:0.0:1787096278.452269:0:6464:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983a68380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:f9858421-28a9-4744-99eb-72e5bedc52c4+20:2406:x1873906077062016:12345-192.168.201.52@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787096278.452276:0:6464:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787096278.452280:0:6464:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89767c00/0x9eebb2eb7f50d8e5 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41131 expref: 20 pid: 8384 timeout: 328 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.452296:0:6464:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89767c00/0x9eebb2eb7f50d8e5 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41131 expref: 20 pid: 8384 timeout: 328 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.452310:0:6464:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89767c00/0x9eebb2eb7f50d8e5 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41131 expref: 20 pid: 8384 timeout: 328 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.452322:0:6464:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3:0x0].0x0 (ffff8c7aadca8100) 00010000:00010000:0.0:1787096278.452327:0:6464:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d8fa lrc: 3/0,1 mode: --/EX res: [0x240000402:0x3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 10275 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.452335:0:6464:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d8fa lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 10275 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.452343:0:6464:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d8fa lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 10275 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096278.452394:0:6464:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787096278.452405:0:10275:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d8fa lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10275 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.452414:0:10275:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d8fa lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 10275 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787096278.452432:0:10275:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x3:0x0]: returning size 0 00000100:00100000:0.0:1787096278.452440:0:6464:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983a68380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:f9858421-28a9-4744-99eb-72e5bedc52c4+19:2406:x1873906077062016:12345-192.168.201.52@tcp:103:ldlm_bl.0 Request processed in 169us (273us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096278.452450:0:6464:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 62 00010000:00010000:2.0:1787096278.452555:0:10275:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d8fa lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 10275 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.452561:0:10275:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d8fa lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 10275 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.452567:0:10275:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d8fa lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10275 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.452570:0:10275:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8c7a89766600 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:2.0:1787096278.452572:0:10275:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d8fa lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 10275 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.452586:0:10275:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d8fa lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 10275 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.452596:0:10275:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3:0x0].0x0 (ffff8c7aadca8100) 00010000:00010000:2.0:1787096278.452598:0:10275:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787096278.452601:0:10275:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d8fa lrc: 2/0,0 mode: --/EX res: [0x240000402:0x3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10275 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.452605:0:10275:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766600/0x9eebb2eb7f50d8fa lrc: 0/0,0 mode: --/EX res: [0x240000402:0x3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 10275 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.452627:0:10275:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766000/0x9eebb2eb7f50d8f3 lrc: 1/0,0 mode: --/CR res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41138 expref: 18 pid: 10275 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.452638:0:10275:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3:0x0].0x0 (ffff8c7aadca8100) 00010000:00010000:2.0:1787096278.452640:0:10275:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787096278.452644:0:10275:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff8c7a89766000/0x9eebb2eb7f50d8f3 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.52@tcp remote: 0x4ef7bc65b7b41138 expref: 18 pid: 10275 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096278.452659:0:10275:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8c7a89766000, rc 0) 00000100:00100000:2.0:1787096278.452695:0:10275:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983a6aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f9858421-28a9-4744-99eb-72e5bedc52c4+17:34322:x1873906077061888:12345-192.168.201.52@tcp:101:lfs.0 Request processed in 1890us (1987us total) trans 0 rc 301/301 00000100:00100000:2.0:1787096278.452700:0:10275:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 173 00000100:00100000:1.0:1787096278.453701:0:3621:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.52@tcp (source: 12345-192.168.201.52@tcp) 00000100:00100000:0.0:1787096278.453744:0:6479:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906077062272 00000100:00100000:0.0:1787096278.453749:0:6479:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906077062272 00000100:00100000:0.0:1787096278.453777:0:6479:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.52@tcp, seq: 26 00000100:00100000:0.0:1787096278.453784:0:6479:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff8c7983a6b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f9858421-28a9-4744-99eb-72e5bedc52c4+17:34322:x1873906077062272:12345-192.168.201.52@tcp:35:lfs.0 00000100:00100000:0.0:1787096278.453897:0:6479:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff8c7983a6b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f9858421-28a9-4744-99eb-72e5bedc52c4+18:34322:x1873906077062272:12345-192.168.201.52@tcp:35:lfs.0 Request processed in 113us (199us total) trans 4294967307 rc 0/0 00000100:00100000:0.0:1787096278.453903:0:6479:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.52@tcp, seq: 26 00010000:00010000:0.0:1787096278.704509:0:8374:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1787096278.931036:0:22009:0:(debug.c:651:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-ec test_23b: @@@@@@ FAIL: setstripe failed Debug log: 1352 lines, 1352 kept, 0 dropped, 0 bad.