00000100:00100000:1.0F:1787096553.870378:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:0.0F:1787096553.870454:0:6481:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352834048 00000100:00100000:0.0:1787096553.870470:0:6481:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352834048 00000100:00100000:0.0:1787096553.870527:0:6481:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 155 00000100:00100000:0.0:1787096553.870540:0:6481:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42f9a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:47c5a664-3f41-4887-ab67-ea7db94c9f90+11:34122:x1873906352834048:12345-192.168.203.27@tcp:101:rm.0 00010000:00010000:0.0:1787096553.870565:0:6481:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096553.870589:0:6481:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccea200/0x1b9fb40e15b43cbd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x8ce29da6e2cc6c0f expref: -99 pid: 6481 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.870664:0:6481:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccea000/0x1b9fb40e15b43cc4 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.870679:0:6481:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccea000/0x1b9fb40e15b43cc4 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.870696:0:6481:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccea000/0x1b9fb40e15b43cc4 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787096553.870781:0:6481:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x1:0x0]: returning size 4096 00000004:00200000:0.0:1787096553.870786:0:6481:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:0.0:1787096553.870819:0:6481:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccea000/0x1b9fb40e15b43cc4 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.870860:0:6481:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccea200/0x1b9fb40e15b43cbd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c0f expref: 13 pid: 6481 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.870881:0:6481:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccea000/0x1b9fb40e15b43cc4 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c0f expref: 12 pid: 6481 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.870890:0:6481:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff9c9d6b3f0700) 00010000:00010000:0.0:1787096553.870893:0:6481:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787096553.870897:0:6481:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d7ccea000, rc 0) 00000100:00100000:0.0:1787096553.870959:0:6481:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42f9a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:47c5a664-3f41-4887-ab67-ea7db94c9f90+12:34122:x1873906352834048:12345-192.168.203.27@tcp:101:rm.0 Request processed in 423us (585us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096553.870969:0:6481:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 155 00000100:00100000:0.0:1787096553.871950:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:1.0:1787096553.871985:0:18819:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352834176 00000100:00100000:1.0:1787096553.871990:0:18819:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352834176 00000100:00100000:1.0:1787096553.872024:0:18819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 156 00000100:00100000:1.0:1787096553.872031:0:18819:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d4800ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:47c5a664-3f41-4887-ab67-ea7db94c9f90+12:34122:x1873906352834176:12345-192.168.203.27@tcp:101:rm.0 00010000:00010000:1.0:1787096553.872041:0:18819:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787096553.872051:0:18819:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cf400/0x1b9fb40e15b43ccb lrc: 2/0,0 mode: --/CR res: [0x240000400:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x8ce29da6e2cc6c16 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.872077:0:18819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9ce600/0x1b9fb40e15b43cd2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18819 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.872084:0:18819:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9ce600/0x1b9fb40e15b43cd2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.872090:0:18819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9ce600/0x1b9fb40e15b43cd2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787096553.872108:0:18819:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000400:0x2:0x0]: returning size 4096 00000004:00200000:1.0:1787096553.872111:0:18819:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787096553.872128:0:18819:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cf400/0x1b9fb40e15b43ccb lrc: 0/0,0 mode: --/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c16 expref: 14 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.872138:0:18819:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9ce600/0x1b9fb40e15b43cd2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c16 expref: 13 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.872146:0:18819:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2:0x0].0x0 (ffff9c9d6b3f0800) 00010000:00010000:1.0:1787096553.872149:0:18819:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787096553.872152:0:18819:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d6e9ce600, rc 0) 00000100:00100000:1.0:1787096553.872188:0:18819:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d4800ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:47c5a664-3f41-4887-ab67-ea7db94c9f90+13:34122:x1873906352834176:12345-192.168.203.27@tcp:101:rm.0 Request processed in 158us (240us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096553.872200:0:18819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 156 00000100:00100000:0.0:1787096553.873092:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:1.0:1787096553.873120:0:7414:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352834432 00000100:00100000:1.0:1787096553.873124:0:7414:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352834432 00000100:00100000:1.0:1787096553.873151:0:7414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 157 00000100:00100000:1.0:1787096553.873156:0:7414:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d4800ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:47c5a664-3f41-4887-ab67-ea7db94c9f90+13:34122:x1873906352834432:12345-192.168.203.27@tcp:101:rm.0 00010000:00010000:1.0:1787096553.873163:0:7414:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787096553.873171:0:7414:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cee00/0x1b9fb40e15b43cd9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x8ce29da6e2cc6c1d expref: -99 pid: 7414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787096553.873237:0:7414:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x1:0x0]: returning size 4096 00010000:00010000:1.0:1787096553.873317:0:7414:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cee00/0x1b9fb40e15b43cd9 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c1d expref: 15 pid: 7414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.873326:0:7414:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c9d48c61d00) 00010000:00010000:1.0:1787096553.873328:0:7414:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787096553.873330:0:7414:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cee00/0x1b9fb40e15b43cd9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c1d expref: 15 pid: 7414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.873338:0:7414:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d6e9cee00, rc 0) 00000100:00100000:1.0:1787096553.873363:0:7414:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d4800ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:47c5a664-3f41-4887-ab67-ea7db94c9f90+14:34122:x1873906352834432:12345-192.168.203.27@tcp:101:rm.0 Request processed in 208us (275us total) trans 4294967302 rc 301/301 00000100:00100000:1.0:1787096553.873373:0:7414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 157 00000100:00100000:1.0:1787096553.875025:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:0.0:1787096553.875064:0:13479:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352834560 00000100:00100000:0.0:1787096553.875071:0:13479:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352834560 00000100:00100000:0.0:1787096553.875131:0:13479:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 22 00000100:00100000:0.0:1787096553.875141:0:13479:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d6b32a850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:47c5a664-3f41-4887-ab67-ea7db94c9f90+14:34122:x1873906352834560:12345-192.168.203.27@tcp:37:rm.0 00000100:00100000:1.0:1787096553.877002:0:13479:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d6b32a850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:47c5a664-3f41-4887-ab67-ea7db94c9f90+14:34122:x1873906352834560:12345-192.168.203.27@tcp:37:rm.0 Request processed in 1860us (1974us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096553.877012:0:13479:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 22 00000100:00100000:2.0F:1787096553.879100:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:1.0:1787096553.879149:0:6482:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352834688 00000100:00100000:1.0:1787096553.879157:0:6482:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352834688 00000100:00100000:1.0:1787096553.879197:0:6482:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 23 00000100:00100000:1.0:1787096553.879207:0:6482:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42d0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:47c5a664-3f41-4887-ab67-ea7db94c9f90+8:34122:x1873906352834688:12345-192.168.203.27@tcp:37:rm.0 00000100:00100000:1.0:1787096553.881202:0:6482:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42d0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:47c5a664-3f41-4887-ab67-ea7db94c9f90+8:34122:x1873906352834688:12345-192.168.203.27@tcp:37:rm.0 Request processed in 1994us (2113us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096553.881211:0:6482:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 23 00000100:00100000:0.0:1787096553.882511:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:1.0:1787096553.882563:0:13479:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352834944 00000100:00100000:1.0:1787096553.882569:0:13479:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352834944 00000100:00100000:1.0:1787096553.882599:0:13479:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 24 00000100:00100000:1.0:1787096553.882608:0:13479:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d4800f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:47c5a664-3f41-4887-ab67-ea7db94c9f90+14:34122:x1873906352834944:12345-192.168.203.27@tcp:35:rm.0 00000100:00100000:1.0:1787096553.882769:0:13479:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d4800f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:47c5a664-3f41-4887-ab67-ea7db94c9f90+15:34122:x1873906352834944:12345-192.168.203.27@tcp:35:rm.0 Request processed in 163us (262us total) trans 4294967303 rc 0/0 00000100:00100000:1.0:1787096553.882775:0:13479:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 24 00000100:00100000:2.0:1787096553.883703:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:0.0:1787096553.883752:0:9995:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352835200 00000100:00100000:0.0:1787096553.883757:0:9995:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352835200 00000100:00100000:0.0:1787096553.883796:0:9995:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 158 00000100:00100000:0.0:1787096553.883804:0:9995:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42d08000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:47c5a664-3f41-4887-ab67-ea7db94c9f90+15:34122:x1873906352835200:12345-192.168.203.27@tcp:101:rm.0 00010000:00010000:0.0:1787096553.883819:0:9995:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096553.883853:0:9995:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccea800/0x1b9fb40e15b43ce7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x8ce29da6e2cc6c24 expref: -99 pid: 9995 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.883960:0:9995:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccebc00/0x1b9fb40e15b43cee lrc: 3/1,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.883974:0:9995:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccebc00/0x1b9fb40e15b43cee lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9995 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.883987:0:9995:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccebc00/0x1b9fb40e15b43cee lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9995 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787096553.883998:0:9995:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x1:0x0]: returning size 4096 00010000:00010000:0.0:1787096553.884056:0:9995:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccea800/0x1b9fb40e15b43ce7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c24 expref: 18 pid: 9995 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.884077:0:9995:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccebc00/0x1b9fb40e15b43cee lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c24 expref: 17 pid: 9995 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.884083:0:9995:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff9c9d6b3f0700) 00010000:00010000:0.0:1787096553.884085:0:9995:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1787096553.884088:0:9995:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d7ccebc00, rc 0) 00000100:00100000:0.0:1787096553.884115:0:9995:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42d08000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:47c5a664-3f41-4887-ab67-ea7db94c9f90+17:34122:x1873906352835200:12345-192.168.203.27@tcp:101:rm.0 Request processed in 314us (414us total) trans 4294967304 rc 0/0 00000100:00100000:0.0:1787096553.884120:0:9995:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 158 00000100:00100000:1.0:1787096553.885256:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:0.0:1787096553.885293:0:6480:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352835328 00000100:00100000:0.0:1787096553.885298:0:6480:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352835328 00000100:00100000:0.0:1787096553.885330:0:6480:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 159 00000100:00100000:0.0:1787096553.885339:0:6480:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42f9b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:47c5a664-3f41-4887-ab67-ea7db94c9f90+8:34122:x1873906352835328:12345-192.168.203.27@tcp:41:rm.0 00000004:00000024:0.0:1787096553.885372:0:6480:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 7847059456 avail 7384014848 00000100:00100000:0.0:1787096553.885444:0:6480:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42f9b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:47c5a664-3f41-4887-ab67-ea7db94c9f90+8:34122:x1873906352835328:12345-192.168.203.27@tcp:41:rm.0 Request processed in 105us (187us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096553.885452:0:6480:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 159 00000100:00100000:1.0:1787096553.886269:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:0.0:1787096553.886309:0:6479:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352835456 00000100:00100000:0.0:1787096553.886314:0:6479:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352835456 00000100:00100000:0.0:1787096553.886347:0:6479:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 160 00000100:00100000:0.0:1787096553.886356:0:6479:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42f9ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:47c5a664-3f41-4887-ab67-ea7db94c9f90+17:34122:x1873906352835456:12345-192.168.203.27@tcp:101:rm.0 00010000:00010000:0.0:1787096553.886368:0:6479:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096553.886381:0:6479:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceac00/0x1b9fb40e15b43cfc lrc: 2/0,0 mode: --/CR res: [0x240000400:0x2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x8ce29da6e2cc6c2b expref: -99 pid: 6479 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886471:0:6479:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccebe00/0x1b9fb40e15b43d03 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6479 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886481:0:6479:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccebe00/0x1b9fb40e15b43d03 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886490:0:6479:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccebe00/0x1b9fb40e15b43d03 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886499:0:6479:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43d0a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6479 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886504:0:6479:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43d0a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886511:0:6479:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43d0a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886567:0:6479:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceb400/0x1b9fb40e15b43d11 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6479 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886575:0:6479:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceb400/0x1b9fb40e15b43d11 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x2:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886580:0:6479:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceb400/0x1b9fb40e15b43d11 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x2:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787096553.886608:0:6479:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x2:0x0]: returning size 0 00000004:00200000:0.0:1787096553.886611:0:6479:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787096553.886623:0:6479:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceb400/0x1b9fb40e15b43d11 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x2:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886640:0:6479:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceb400/0x1b9fb40e15b43d11 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886647:0:6479:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccebe00/0x1b9fb40e15b43d03 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886654:0:6479:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccebe00/0x1b9fb40e15b43d03 lrc: 2/0,0 mode: CR/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886660:0:6479:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccebe00/0x1b9fb40e15b43d03 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886666:0:6479:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d7ccebe00 already unused, calling callback (ffffffffc19474f0) 00010000:00010000:0.0:1787096553.886669:0:6479:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccebe00/0x1b9fb40e15b43d03 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886677:0:6479:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccebe00/0x1b9fb40e15b43d03 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886687:0:6479:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2:0x0].0x0 (ffff9c9d6b3f0800) 00010000:00010000:0.0:1787096553.886691:0:6479:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096553.886694:0:6479:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccebe00/0x1b9fb40e15b43d03 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886699:0:6479:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccebe00/0x1b9fb40e15b43d03 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886708:0:6479:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43d0a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886714:0:6479:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43d0a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886724:0:6479:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43d0a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886729:0:6479:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d7cceae00 already unused, calling callback (ffffffffc19474f0) 00010000:00010000:0.0:1787096553.886732:0:6479:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43d0a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886740:0:6479:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43d0a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886749:0:6479:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2:0x0].0x7b711ae6 (ffff9c9d48c61800) 00010000:00010000:0.0:1787096553.886752:0:6479:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096553.886755:0:6479:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43d0a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886761:0:6479:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43d0a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886785:0:6479:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceac00/0x1b9fb40e15b43cfc lrc: 0/0,0 mode: --/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c2b expref: 19 pid: 6479 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886794:0:6479:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceb400/0x1b9fb40e15b43d11 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c2b expref: 18 pid: 6479 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.886799:0:6479:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x2:0x0].0x0 (ffff9c9d6b35c000) 00010000:00010000:0.0:1787096553.886801:0:6479:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787096553.886803:0:6479:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d7cceb400, rc 0) 00000100:00100000:0.0:1787096553.886845:0:6479:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42f9ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:47c5a664-3f41-4887-ab67-ea7db94c9f90+18:34122:x1873906352835456:12345-192.168.203.27@tcp:101:rm.0 Request processed in 478us (563us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096553.886857:0:6479:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 160 00000100:00100000:1.0:1787096553.888029:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:2.0:1787096553.888079:0:6481:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352835712 00000100:00100000:2.0:1787096553.888085:0:6481:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352835712 00000100:00100000:2.0:1787096553.888121:0:6481:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 161 00000100:00100000:2.0:1787096553.888130:0:6481:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d57c68700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:47c5a664-3f41-4887-ab67-ea7db94c9f90+18:34122:x1873906352835712:12345-192.168.203.27@tcp:36:rm.0 00010000:00010000:2.0:1787096553.888160:0:6481:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1787096553.888176:0:6481:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7cceb400/0x1b9fb40e15b43d11 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c2b expref: 18 pid: 6479 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.888189:0:6481:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x2:0x0].0x0 (ffff9c9d6b35c000) 00010000:00010000:2.0:1787096553.888204:0:6481:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9ce600/0x1b9fb40e15b43cd2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c16 expref: 17 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.888213:0:6481:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2:0x0].0x0 (ffff9c9d6b3f0800) 00010000:00010000:2.0:1787096553.888217:0:6481:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787096553.888229:0:6481:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4be00/0x1b9fb40e15b43d18 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.888238:0:6481:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4be00/0x1b9fb40e15b43d18 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889485:0:6481:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4be00/0x1b9fb40e15b43d18 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889498:0:6481:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b000/0x1b9fb40e15b43d1f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889504:0:6481:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b000/0x1b9fb40e15b43d1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889521:0:6481:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b000/0x1b9fb40e15b43d1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889587:0:6481:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4bc00/0x1b9fb40e15b43d26 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889593:0:6481:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4bc00/0x1b9fb40e15b43d26 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889598:0:6481:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4bc00/0x1b9fb40e15b43d26 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889752:0:6481:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4aa00/0x1b9fb40e15b43d2d lrc: 3/0,1 mode: --/PW res: [0x240000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889761:0:6481:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4aa00/0x1b9fb40e15b43d2d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889768:0:6481:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4aa00/0x1b9fb40e15b43d2d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889775:0:6481:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4aa00/0x1b9fb40e15b43d2d lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889781:0:6481:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4aa00/0x1b9fb40e15b43d2d lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889787:0:6481:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4aa00/0x1b9fb40e15b43d2d lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889810:0:6481:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787096553.889826:0:6468:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787096553.889842:0:6481:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4bc00/0x1b9fb40e15b43d26 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.889844:0:6468:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4aa00/0x1b9fb40e15b43d2d lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889860:0:6481:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4bc00/0x1b9fb40e15b43d26 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889867:0:6481:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4bc00/0x1b9fb40e15b43d26 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.889867:0:6468:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d7ca4aa00 already unused, calling callback (ffffffffc0dbb6f0) 00010000:00010000:0.0:1787096553.889873:0:6468:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4aa00/0x1b9fb40e15b43d2d lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889878:0:6481:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d7ca4bc00 already unused, calling callback (ffffffffc19474f0) 00010000:00010000:0.0:1787096553.889882:0:6468:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4aa00/0x1b9fb40e15b43d2d lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889883:0:6481:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4bc00/0x1b9fb40e15b43d26 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.889899:0:6468:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x2:0x0].0x0 (ffff9c9d6b340000) 00010000:00010000:2.0:1787096553.889900:0:6481:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4bc00/0x1b9fb40e15b43d26 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.889903:0:6468:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787096553.889908:0:6468:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4aa00/0x1b9fb40e15b43d2d lrc: 1/0,0 mode: --/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.889913:0:6468:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4aa00/0x1b9fb40e15b43d2d lrc: 0/0,0 mode: --/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889914:0:6481:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x2:0x0].0x0 (ffff9c9d6b340000) 00010000:00010000:2.0:1787096553.889921:0:6481:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787096553.889926:0:6481:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4bc00/0x1b9fb40e15b43d26 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.889927:0:6468:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096553.889929:0:6468:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096553.889932:0:6468:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787096553.889936:0:6481:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4bc00/0x1b9fb40e15b43d26 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889955:0:6481:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4be00/0x1b9fb40e15b43d18 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889961:0:6481:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4be00/0x1b9fb40e15b43d18 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889968:0:6481:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4be00/0x1b9fb40e15b43d18 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.889974:0:6481:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d7ca4be00 already unused, calling callback (ffffffffc19474f0) 00010000:00010000:2.0:1787096553.890667:0:6481:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4be00/0x1b9fb40e15b43d18 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.890675:0:6481:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4be00/0x1b9fb40e15b43d18 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.890685:0:6481:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2:0x0].0x0 (ffff9c9d6b3f0800) 00010000:00010000:2.0:1787096553.890688:0:6481:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096553.890691:0:6481:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4be00/0x1b9fb40e15b43d18 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.890698:0:6481:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4be00/0x1b9fb40e15b43d18 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787096553.890709:0:6481:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000b4bc9782 reply state 00000000263400ad transno 4294967305 ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b000/0x1b9fb40e15b43d1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.890720:0:6481:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b000/0x1b9fb40e15b43d1f lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096553.890727:0:6481:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2:0x0].0x7b711ae6 (ffff9c9d48c61800) 00010000:00010000:2.0:1787096553.890729:0:6481:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787096553.890733:0:6481:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787096553.890807:0:6481:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d57c68700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:47c5a664-3f41-4887-ab67-ea7db94c9f90+18:34122:x1873906352835712:12345-192.168.203.27@tcp:36:rm.0 Request processed in 2675us (2776us total) trans 4294967305 rc 0/0 00000100:00100000:2.0:1787096553.890817:0:6481:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 161 00000100:00100000:1.0:1787096553.892236:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:1.0:1787096553.892274:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:0.0:1787096553.892276:0:8378:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352835840 00000100:00100000:0.0:1787096553.892284:0:8378:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352835840 00000100:00100000:3.0F:1787096553.892314:0:6482:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352836096 00000100:00100000:3.0:1787096553.892325:0:6482:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352836096 00000100:00100000:0.0:1787096553.892332:0:8378:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 4 00000100:00100000:0.0:1787096553.892343:0:8378:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d57c26300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:47c5a664-3f41-4887-ab67-ea7db94c9f90+8:2402:x1873906352835840:12345-192.168.203.27@tcp:103:rm.0 00010000:00010000:0.0:1787096553.892359:0:8378:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787096553.892368:0:6482:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 25 00000100:00100000:3.0:1787096553.892378:0:6482:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d74859180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:47c5a664-3f41-4887-ab67-ea7db94c9f90+18:34122:x1873906352836096:12345-192.168.203.27@tcp:35:rm.0 00010000:00010000:0.0:1787096553.892381:0:8378:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceaa00/0x1b9fb40e15b43c38 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6bde expref: 8 pid: 6479 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.892449:0:8378:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff9c9d6b35cc00) 00010000:00010000:0.0:1787096553.892461:0:8378:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787096553.892512:0:8378:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d57c26300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:47c5a664-3f41-4887-ab67-ea7db94c9f90+7:2402:x1873906352835840:12345-192.168.203.27@tcp:103:rm.0 Request processed in 171us (278us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096553.892520:0:8378:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 4 00000100:00100000:3.0:1787096553.892570:0:6482:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d74859180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:47c5a664-3f41-4887-ab67-ea7db94c9f90+19:34122:x1873906352836096:12345-192.168.203.27@tcp:35:rm.0 Request processed in 193us (294us total) trans 4294967306 rc 0/0 00000100:00100000:3.0:1787096553.892578:0:6482:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 25 00000100:00100000:1.0:1787096553.893430:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:3.0:1787096553.893463:0:18819:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352836352 00000100:00100000:3.0:1787096553.893470:0:18819:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352836352 00000100:00100000:3.0:1787096553.893506:0:18819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 162 00000100:00100000:3.0:1787096553.893515:0:18819:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42ebd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:47c5a664-3f41-4887-ab67-ea7db94c9f90+19:34122:x1873906352836352:12345-192.168.203.27@tcp:36:rm.0 00010000:00010000:3.0:1787096553.893546:0:18819:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1787096553.893558:0:18819:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccebc00/0x1b9fb40e15b43cee lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c24 expref: 19 pid: 9995 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.893570:0:18819:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ccea000/0x1b9fb40e15b43cc4 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c0f expref: 18 pid: 6481 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.893580:0:18819:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b600/0x1b9fb40e15b43c46 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6be5 expref: 17 pid: 6481 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.893589:0:18819:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff9c9d6b3f0700) 00010000:00010000:3.0:1787096553.893593:0:18819:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787096553.893633:0:18819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 18819 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.893642:0:18819:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.893663:0:18819:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096553.893684:0:18819:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d7dad4380 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:18819:1873906360244864:0@lo:101:mdt00_005.0 00000100:00100000:3.0:1787096553.893716:0:18819:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096553.893754:0:18819:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c41dad780 going to sleep for 11 seconds 00000100:00100000:1.0:1787096553.893756:0:7723:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360244864 00000100:00100000:1.0:1787096553.893761:0:7723:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360244864 00000100:00100000:1.0:1787096553.893795:0:7723:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 142 00000100:00100000:1.0:1787096553.893804:0:7723:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42ccc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:18819:x1873906360244864:12345-0@lo:101:mdt00_005.0 00010000:00010000:1.0:1787096553.893814:0:7723:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787096553.893827:0:7723:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cf200/0x1b9fb40e15b43d3b lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x1b9fb40e15b43d34 expref: -99 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.893870:0:7723:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cfe00/0x1b9fb40e15b43c1c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6bd7 expref: 6 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.893881:0:7723:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cf200/0x1b9fb40e15b43d3b lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d34 expref: 7 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.893904:0:7723:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cfe00/0x1b9fb40e15b43c1c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6bd7 expref: 6 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.893917:0:7723:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cfe00/0x1b9fb40e15b43c1c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6bd7 expref: 6 pid: 6480 timeout: 329 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096553.893928:0:7723:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d74839180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:7723:1873906360244992:192.168.203.27@tcp:104: 00000100:00100000:1.0:1787096553.893979:0:7723:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c41c79cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787096553.894897:0:7723:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d74839180 pname:cluuid:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000_UUID:7723:1873906360244992:192.168.203.27@tcp:104: 00010000:00010000:0.0:1787096553.894921:0:7723:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cf200/0x1b9fb40e15b43d3b lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d34 expref: 7 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.894929:0:7723:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c9c41286000) 00010000:00010000:0.0:1787096553.894932:0:7723:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096553.894936:0:7723:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cf200/0x1b9fb40e15b43d3b lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d34 expref: 7 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.894945:0:7723:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d6e9cf200, rc 0) 00000100:00100000:0.0:1787096553.894990:0:7723:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42ccc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:18819:x1873906360244864:12345-0@lo:101:mdt00_005.0 Request processed in 1187us (1270us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096553.894997:0:7723:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 142 00000100:00100000:3.0:1787096553.895009:0:18819:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d7dad4380 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:18819:1873906360244864:0@lo:101:mdt00_005.0 00010000:00010000:3.0:1787096553.895026:0:18819:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.895037:0:18819:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096553.895049:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:0.0:1787096553.895102:0:6466:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352836480 00000100:00100000:0.0:1787096553.895107:0:6466:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352836480 00010000:00010000:0.0:1787096553.895130:0:6466:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cfe00/0x1b9fb40e15b43c1c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6bd7 expref: 7 pid: 6480 timeout: 329 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096553.895161:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 63 00000100:00100000:0.0:1787096553.895170:0:6466:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d74839180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:47c5a664-3f41-4887-ab67-ea7db94c9f90+7:2401:x1873906352836480:12345-192.168.203.27@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787096553.895188:0:6466:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787096553.895194:0:6466:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cfe00/0x1b9fb40e15b43c1c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6bd7 expref: 7 pid: 6480 timeout: 329 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.895207:0:6466:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cfe00/0x1b9fb40e15b43c1c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6bd7 expref: 7 pid: 6480 timeout: 329 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.895223:0:6466:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cfe00/0x1b9fb40e15b43c1c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6bd7 expref: 7 pid: 6480 timeout: 329 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.895236:0:6466:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c9c41286000) 00010000:00010000:0.0:1787096553.895241:0:6466:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cf200/0x1b9fb40e15b43d3b lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d34 expref: 6 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.895251:0:6466:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cf200/0x1b9fb40e15b43d3b lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x1b9fb40e15b43d34 expref: 6 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.895260:0:6466:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cf200/0x1b9fb40e15b43d3b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x1b9fb40e15b43d34 expref: 6 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.895281:0:6466:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cf200/0x1b9fb40e15b43d3b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d34 expref: 6 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096553.895291:0:6466:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d4800f800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6466:1873906360245120:0@lo:105: 00000100:00100000:0.0:1787096553.895320:0:6466:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096553.895348:0:6464:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360245120 00000100:00100000:0.0:1787096553.895352:0:6466:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c42e45f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787096553.895353:0:6464:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360245120 00000100:00100000:1.0:1787096553.895389:0:6464:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 43 00000100:00100000:1.0:1787096553.895422:0:6464:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42dc8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6466:x1873906360245120:12345-0@lo:105: 00010000:00010000:1.0:1787096553.895432:0:6464:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.895442:0:6464:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.895451:0:6464:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.895467:0:6464:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.895475:0:6464:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.895495:0:6464:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.895517:0:6464:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9c9d6eac3400) 00010000:00010000:3.0:1787096553.895526:0:18819:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.895536:0:18819:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787096553.895555:0:18819:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:1.0:1787096553.895562:0:6464:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42dc8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6466:x1873906360245120:12345-0@lo:105: Request processed in 142us (240us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096553.895574:0:6464:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 43 00000100:00100000:0.0:1787096553.895574:0:6466:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d4800f800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6466:1873906360245120:0@lo:105: 00010000:00010000:3.0:1787096553.895584:0:18819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac2600/0x1b9fb40e15b43d42 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 18819 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.895592:0:18819:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac2600/0x1b9fb40e15b43d42 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.895598:0:6466:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787096553.895600:0:18819:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac2600/0x1b9fb40e15b43d42 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096553.895624:0:18819:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9c42ccdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:18819:1873906360245248:0@lo:101:mdt00_005.0 00000100:00100000:3.0:1787096553.895662:0:18819:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096553.895663:0:6466:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d74839180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:47c5a664-3f41-4887-ab67-ea7db94c9f90+6:2401:x1873906352836480:12345-192.168.203.27@tcp:103:ldlm_bl.0 Request processed in 491us (614us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096553.895673:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 63 00000100:00100000:3.0:1787096553.895684:0:18819:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c41dad780 going to sleep for 11 seconds 00000100:00100000:1.0:1787096553.895741:0:7409:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360245248 00000100:00100000:1.0:1787096553.895747:0:7409:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360245248 00000100:00100000:1.0:1787096553.895784:0:7409:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 143 00000100:00100000:1.0:1787096553.895794:0:7409:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d7dad4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:18819:x1873906360245248:12345-0@lo:101:mdt00_005.0 00010000:00010000:1.0:1787096553.895807:0:7409:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787096553.895820:0:7409:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9ce600/0x1b9fb40e15b43d49 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x1b9fb40e15b43d42 expref: -99 pid: 7409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.895852:0:7409:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9ce600/0x1b9fb40e15b43d49 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d42 expref: 8 pid: 7409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.895863:0:7409:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9ce600/0x1b9fb40e15b43d49 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d42 expref: 8 pid: 7409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.895882:0:7409:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa5d0027 (ffff9c9d48c61d00) 00010000:00010000:1.0:1787096553.895886:0:7409:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787096553.895889:0:7409:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d6e9ce600, rc 0) 00000100:00100000:1.0:1787096553.895925:0:7409:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d7dad4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:18819:x1873906360245248:12345-0@lo:101:mdt00_005.0 Request processed in 133us (264us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096553.895932:0:7409:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 143 00000100:00100000:3.0:1787096553.895937:0:18819:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9c42ccdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:18819:1873906360245248:0@lo:101:mdt00_005.0 00010000:00010000:3.0:1787096553.895950:0:18819:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac2600/0x1b9fb40e15b43d42 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x1b9fb40e15b43d49 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.895960:0:18819:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac2600/0x1b9fb40e15b43d42 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43d49 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787096553.895972:0:18819:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1787096553.896009:0:18819:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9c42ccdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:18819:1873906360245376:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787096553.896039:0:18819:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096553.896058:0:18819:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c41dad000 going to sleep for 11 seconds 00000100:00100000:0.0:1787096553.896069:0:6484:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360245376 00000100:00100000:0.0:1787096553.896073:0:6484:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360245376 00000100:00100000:0.0:1787096553.896099:0:6484:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 144 00000100:00100000:0.0:1787096553.896104:0:6484:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d748eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:18819:x1873906360245376:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787096553.896157:0:6484:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d748eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:18819:x1873906360245376:12345-0@lo:1000:mdt00_005.0 Request processed in 53us (118us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096553.896162:0:6484:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 144 00000100:00100000:3.0:1787096553.896173:0:18819:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9c42ccdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:18819:1873906360245376:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787096553.896202:0:18819:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d748ece00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:18819:1873906360245504:0@lo:1000:mdt00_005.0 00000100:00100000:3.0:1787096553.896219:0:18819:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096553.896231:0:18819:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c41dad000 going to sleep for 11 seconds 00000100:00100000:0.0:1787096553.896250:0:6485:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360245504 00000100:00100000:0.0:1787096553.896255:0:6485:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360245504 00000100:00100000:0.0:1787096553.896280:0:6485:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 145 00000100:00100000:0.0:1787096553.896288:0:6485:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d748ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:18819:x1873906360245504:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787096553.896363:0:6485:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d748ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:18819:x1873906360245504:12345-0@lo:1000:mdt00_005.0 Request processed in 76us (141us total) trans 0 rc 1/1 00000100:00100000:3.0:1787096553.896369:0:18819:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d748ece00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:18819:1873906360245504:0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787096553.896374:0:6485:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 145 00010000:00010000:3.0:1787096553.896531:0:18819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3800/0x1b9fb40e15b43d50 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 18819 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.896537:0:18819:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3800/0x1b9fb40e15b43d50 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.896541:0:18819:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3800/0x1b9fb40e15b43d50 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096553.896553:0:18819:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d748ece00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:18819:1873906360245632:0@lo:101:mdt00_005.0 00000100:00100000:3.0:1787096553.896569:0:18819:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096553.896581:0:18819:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c41dad780 going to sleep for 11 seconds 00000100:00100000:1.0:1787096553.896594:0:7723:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360245632 00000100:00100000:1.0:1787096553.896598:0:7723:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360245632 00000100:00100000:1.0:1787096553.896627:0:7723:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 146 00000100:00100000:1.0:1787096553.896636:0:7723:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42ccdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:18819:x1873906360245632:12345-0@lo:101:mdt00_005.0 00010000:00010000:1.0:1787096553.896645:0:7723:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787096553.896656:0:7723:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cf800/0x1b9fb40e15b43d57 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x1b9fb40e15b43d50 expref: -99 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.896671:0:7723:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cf800/0x1b9fb40e15b43d57 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d50 expref: 9 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.896678:0:7723:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cf800/0x1b9fb40e15b43d57 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d50 expref: 9 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.896683:0:7723:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff9c9d48c61500) 00010000:00010000:1.0:1787096553.896684:0:7723:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787096553.896686:0:7723:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d6e9cf800, rc 0) 00000100:00100000:1.0:1787096553.896713:0:7723:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42ccdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:18819:x1873906360245632:12345-0@lo:101:mdt00_005.0 Request processed in 80us (142us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096553.896718:0:7723:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 146 00000100:00100000:3.0:1787096553.896724:0:18819:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d748ece00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:18819:1873906360245632:0@lo:101:mdt00_005.0 00010000:00010000:3.0:1787096553.896736:0:18819:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3800/0x1b9fb40e15b43d50 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x1b9fb40e15b43d57 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.896742:0:18819:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3800/0x1b9fb40e15b43d50 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43d57 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.896756:0:18819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2c00/0x1b9fb40e15b43d5e lrc: 3/0,1 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18819 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.896761:0:18819:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2c00/0x1b9fb40e15b43d5e lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.896766:0:18819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2c00/0x1b9fb40e15b43d5e lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.896777:0:18819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2400/0x1b9fb40e15b43d65 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18819 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.896781:0:18819:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2400/0x1b9fb40e15b43d65 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.896784:0:18819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2400/0x1b9fb40e15b43d65 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.896795:0:18819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 18819 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.896803:0:18819:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.896808:0:18819:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096553.896814:0:18819:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d748ece00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:18819:1873906360245760:0@lo:101:mdt00_005.0 00000100:00100000:3.0:1787096553.896846:0:18819:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096553.896858:0:18819:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c41dad780 going to sleep for 11 seconds 00000100:00100000:1.0:1787096553.896863:0:7409:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360245760 00000100:00100000:1.0:1787096553.896866:0:7409:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360245760 00000100:00100000:1.0:1787096553.896893:0:7409:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 147 00000100:00100000:1.0:1787096553.896900:0:7409:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d6ea18380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:18819:x1873906360245760:12345-0@lo:101:mdt00_005.0 00010000:00010000:1.0:1787096553.896908:0:7409:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787096553.896917:0:7409:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cfc00/0x1b9fb40e15b43d73 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x1b9fb40e15b43d6c expref: -99 pid: 7409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.896934:0:7409:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9ce800/0x1b9fb40e15b43c62 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6bf3 expref: 5 pid: 7414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.896943:0:7409:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cfc00/0x1b9fb40e15b43d73 lrc: 4/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d6c expref: 10 pid: 7409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.896973:0:7409:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9ce800/0x1b9fb40e15b43c62 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6bf3 expref: 5 pid: 7414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.896986:0:7409:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9ce800/0x1b9fb40e15b43c62 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6bf3 expref: 5 pid: 7414 timeout: 329 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096553.896997:0:7409:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d7bf40e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:7409:1873906360245888:192.168.203.27@tcp:104: 00000100:00100000:1.0:1787096553.897035:0:7409:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c41c79e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787096553.898162:0:7409:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d7bf40e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:7409:1873906360245888:192.168.203.27@tcp:104: 00000100:00100000:1.0:1787096553.898179:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00010000:00010000:3.0:1787096553.898181:0:7409:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cfc00/0x1b9fb40e15b43d73 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d6c expref: 10 pid: 7409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.898188:0:7409:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2:0x0].0x0 (ffff9c9c41286b00) 00010000:00010000:3.0:1787096553.898190:0:7409:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787096553.898193:0:7409:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cfc00/0x1b9fb40e15b43d73 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d6c expref: 10 pid: 7409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.898199:0:7409:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cfc00/0x1b9fb40e15b43d73 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d6c expref: 10 pid: 7409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.898204:0:7409:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d6e9cfc00, rc 0) 00000100:00100000:0.0:1787096553.898217:0:9435:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352836608 00000100:00100000:0.0:1787096553.898222:0:9435:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352836608 00010000:00010000:0.0:1787096553.898240:0:9435:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9ce800/0x1b9fb40e15b43c62 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6bf3 expref: 6 pid: 7414 timeout: 329 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096553.898249:0:18819:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d748ece00 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:18819:1873906360245760:0@lo:101:mdt00_005.0 00010000:00010000:1.0:1787096553.898260:0:18819:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 4/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096553.898266:0:9435:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 64 00010000:00010000:1.0:1787096553.898269:0:18819:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 4/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096553.898274:0:9435:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d62798050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:47c5a664-3f41-4887-ab67-ea7db94c9f90+6:2401:x1873906352836608:12345-192.168.203.27@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787096553.898280:0:7409:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d6ea18380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:18819:x1873906360245760:12345-0@lo:101:mdt00_005.0 Request processed in 1376us (1430us total) trans 0 rc 0/0 00010000:00010000:0.0:1787096553.898289:0:9435:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787096553.898295:0:7409:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 147 00010000:00010000:0.0:1787096553.898296:0:9435:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9ce800/0x1b9fb40e15b43c62 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6bf3 expref: 6 pid: 7414 timeout: 329 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.898313:0:9435:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9ce800/0x1b9fb40e15b43c62 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6bf3 expref: 6 pid: 7414 timeout: 329 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.898324:0:9435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9ce800/0x1b9fb40e15b43c62 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6bf3 expref: 6 pid: 7414 timeout: 329 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.898331:0:9435:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2:0x0].0x0 (ffff9c9c41286b00) 00010000:00010000:0.0:1787096553.898335:0:9435:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cfc00/0x1b9fb40e15b43d73 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d6c expref: 9 pid: 7409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.898340:0:9435:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cfc00/0x1b9fb40e15b43d73 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x1b9fb40e15b43d6c expref: 9 pid: 7409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.898346:0:9435:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cfc00/0x1b9fb40e15b43d73 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x1b9fb40e15b43d6c expref: 9 pid: 7409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.898365:0:9435:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cfc00/0x1b9fb40e15b43d73 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d6c expref: 9 pid: 7409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096553.898376:0:9435:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9c42dc9500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:9435:1873906360246016:0@lo:105: 00000100:00100000:0.0:1787096553.898431:0:9435:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096553.898455:0:8375:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360246016 00000100:00100000:0.0:1787096553.898457:0:9435:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c42e45f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787096553.898459:0:8375:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360246016 00000100:00100000:3.0:1787096553.898488:0:8375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 44 00000100:00100000:3.0:1787096553.898492:0:8375:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42dc8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:lustre-MDT0001-mdtlov_UUID+4:9435:x1873906360246016:12345-0@lo:105: 00010000:00010000:3.0:1787096553.898499:0:8375:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 5/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.898510:0:8375:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 5/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.898515:0:8375:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 5/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.898520:0:8375:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 5/0,1 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.898524:0:8375:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 6/0,1 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.898528:0:8375:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 6/0,1 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.898541:0:8375:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9c9d6eac3e00) 00010000:00010000:1.0:1787096553.898559:0:18819:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.898566:0:18819:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096553.898569:0:8375:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42dc8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:lustre-MDT0001-mdtlov_UUID+4:9435:x1873906360246016:12345-0@lo:105: Request processed in 76us (136us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096553.898580:0:8375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 44 00000100:00100000:0.0:1787096553.898590:0:9435:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9c42dc9500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0000_UUID:9435:1873906360246016:0@lo:105: 00010000:00010000:0.0:1787096553.898601:0:9435:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787096553.898632:0:9435:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d62798050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:47c5a664-3f41-4887-ab67-ea7db94c9f90+5:2401:x1873906352836608:12345-192.168.203.27@tcp:103:ldlm_bl.0 Request processed in 359us (455us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096553.898650:0:9435:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 64 00000100:00100000:1.0:1787096553.899157:0:18819:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d7bf42300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:18819:1873906360246144:0@lo:403:mdt00_005.0 00000100:00100000:1.0:1787096553.899185:0:18819:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096553.899202:0:18819:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c41c79e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787096553.899214:0:6484:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360246144 00000100:00100000:0.0:1787096553.899218:0:6484:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360246144 00000100:00100000:0.0:1787096553.899246:0:6484:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 148 00000100:00100000:0.0:1787096553.899254:0:6484:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d7bf42d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:18819:x1873906360246144:12345-0@lo:403:mdt00_005.0 00000100:00100000:0.0:1787096553.900746:0:6484:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d7bf42d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:18819:x1873906360246144:12345-0@lo:403:mdt00_005.0 Request processed in 1493us (1559us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096553.900752:0:6484:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 148 00000100:00100000:1.0:1787096553.900783:0:18819:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d7bf42300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:18819:1873906360246144:0@lo:403:mdt00_005.0 00000040:00080000:1.0:1787096553.901629:0:18819:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].2, 936 off35856 00000100:00100000:1.0:1787096553.901880:0:18819:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d7bf42300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:18819:1873906360246272:0@lo:1000:mdt00_005.0 00000100:00100000:1.0:1787096553.901918:0:18819:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096553.901947:0:18819:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c41c79a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787096553.901966:0:6485:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360246272 00000100:00100000:0.0:1787096553.901973:0:6485:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360246272 00000100:00100000:0.0:1787096553.902017:0:6485:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 149 00000100:00100000:0.0:1787096553.902026:0:6485:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d7bf40e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:18819:x1873906360246272:12345-0@lo:1000:mdt00_005.0 00000100:00100000:0.0:1787096553.902099:0:6485:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d7bf40e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:18819:x1873906360246272:12345-0@lo:1000:mdt00_005.0 Request processed in 74us (180us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096553.902106:0:6485:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 149 00000100:00100000:1.0:1787096553.902129:0:18819:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d7bf42300 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-mdtlov_UUID:18819:1873906360246272:0@lo:1000:mdt00_005.0 00000040:00080000:1.0:1787096553.902159:0:18819:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].2, 1016 off36136 00000004:00080000:1.0:1787096553.902167:0:18819:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c9c41c79e40 with version 5 00000004:00080000:0.0:1787096553.902205:0:7407:0:(osp_trans.c:1330:osp_get_next_request()) ou ffff9c9d589526c0 version 5 rpc_version 5 00000020:00100000:0.0:1787096553.902228:0:7407:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787096553.902236:0:7407:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c9c42dc9500 x1873906360246528/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2592/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787096553.902252:0:7407:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9c42dc9500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7407:1873906360246528:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787096553.902282:0:7407:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096553.902301:0:7407:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c42e45240 going to sleep for 11 seconds 00000100:00100000:1.0:1787096553.902304:0:7723:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360246528 00000100:00100000:1.0:1787096553.902309:0:7723:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360246528 00000100:00100000:1.0:1787096553.902340:0:7723:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 150 00000100:00100000:1.0:1787096553.902349:0:7723:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:7407:x1873906360246528:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787096553.902714:0:7723:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:7407:x1873906360246528:12345-0@lo:1000:osp_up0-1.0 Request processed in 366us (431us total) trans 4294967359 rc 0/0 00000100:00100000:1.0:1787096553.902723:0:7723:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 150 00000100:00100000:0.0:1787096553.902762:0:7407:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9c42dc9500 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7407:1873906360246528:0@lo:1000:osp_up0-1.0 00000004:00200000:1.0:1787096553.902779:0:18819:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x1:0x0]: returning size 4096 00010000:00010000:1.0:1787096553.902800:0:18819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2400/0x1b9fb40e15b43d65 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.902810:0:18819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2400/0x1b9fb40e15b43d65 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.902818:0:18819:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2400/0x1b9fb40e15b43d65 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.902824:0:18819:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d6eac2400 already unused, calling callback (ffffffffc19474f0) 00010000:00010000:1.0:1787096553.902829:0:18819:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2400/0x1b9fb40e15b43d65 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.902854:0:18819:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2400/0x1b9fb40e15b43d65 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.902870:0:18819:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2:0x0].0x0 (ffff9c9d6b3f0800) 00010000:00010000:1.0:1787096553.902874:0:18819:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787096553.902878:0:18819:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2400/0x1b9fb40e15b43d65 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.902884:0:18819:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2400/0x1b9fb40e15b43d65 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.902899:0:18819:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.902908:0:18819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.902916:0:18819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.902945:0:18819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787096553.902953:0:18819:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000401:0x2:0x0] 00010000:00010000:1.0:1787096553.902962:0:18819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2c00/0x1b9fb40e15b43d5e lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.902969:0:6469:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787096553.902970:0:18819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2c00/0x1b9fb40e15b43d5e lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.902979:0:18819:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2c00/0x1b9fb40e15b43d5e lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.902979:0:6469:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.902986:0:18819:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d6eac2c00 already unused, calling callback (ffffffffc19474f0) 00010000:00010000:1.0:1787096553.902990:0:18819:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2c00/0x1b9fb40e15b43d5e lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.903004:0:18819:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2c00/0x1b9fb40e15b43d5e lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.903015:0:18819:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff9c9d6b3f0700) 00010000:00010000:0.0:1787096553.903015:0:6469:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d6eac3e00 already unused, calling callback (ffffffffc1953ee0) 00010000:00010000:1.0:1787096553.903021:0:18819:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096553.903022:0:6469:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.903024:0:18819:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2c00/0x1b9fb40e15b43d5e lrc: 2/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.903041:0:18819:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6eac2c00/0x1b9fb40e15b43d5e lrc: 0/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787096553.903043:0:6469:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.903056:0:18819:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3800/0x1b9fb40e15b43d50 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43d57 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.903065:0:18819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3800/0x1b9fb40e15b43d50 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x1b9fb40e15b43d57 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.903077:0:6469:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.903079:0:18819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3800/0x1b9fb40e15b43d50 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x1b9fb40e15b43d57 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.903097:0:6469:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787096553.903098:0:18819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787096553.903109:0:18819:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787096553.903114:0:18819:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.903122:0:18819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.903122:0:6469:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 1/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.903139:0:8376:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787096553.903144:0:18819:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.903145:0:6469:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3e00/0x1b9fb40e15b43d6c lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x1b9fb40e15b43d73 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.903147:0:8376:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3800/0x1b9fb40e15b43d50 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x1b9fb40e15b43d57 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.903168:0:8376:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d6eac3800 already unused, calling callback (ffffffffc1953ee0) 00010000:00010000:1.0:1787096553.903177:0:18819:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1787096553.903179:0:8376:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3800/0x1b9fb40e15b43d50 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x1b9fb40e15b43d57 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.903180:0:6469:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000004:00080000:1.0:1787096553.903182:0:18819:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787096553.903190:0:18819:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac2600/0x1b9fb40e15b43d42 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x1b9fb40e15b43d49 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787096553.903192:0:8376:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3800/0x1b9fb40e15b43d50 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x1b9fb40e15b43d57 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096553.903209:0:18819:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac2600/0x1b9fb40e15b43d42 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x1b9fb40e15b43d49 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.903225:0:8376:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3800/0x1b9fb40e15b43d50 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x1b9fb40e15b43d57 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.903232:0:8376:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787096553.903240:0:8376:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3800/0x1b9fb40e15b43d50 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x1b9fb40e15b43d57 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.903243:0:6469:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.903249:0:8376:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3800/0x1b9fb40e15b43d50 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x1b9fb40e15b43d57 expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.903258:0:6469:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d6eac3400 already unused, calling callback (ffffffffc1953ee0) 00010000:00010000:0.0:1787096553.903264:0:6469:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096553.903266:0:18819:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42ebd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:47c5a664-3f41-4887-ab67-ea7db94c9f90+17:34122:x1873906352836352:12345-192.168.203.27@tcp:36:rm.0 Request processed in 9750us (9834us total) trans 4294967307 rc 0/0 00010000:00010000:3.0:1787096553.903269:0:8376:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787096553.903273:0:8376:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787096553.903277:0:8376:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787096553.903282:0:18819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 162 00000004:00010000:0.0:1787096553.903284:0:6469:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096553.903287:0:8377:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096553.903317:0:6469:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.903324:0:6469:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787096553.903346:0:6469:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096553.903353:0:3627:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9c42dcb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3627:1873906360246912:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787096553.903356:0:6469:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6eac3400/0x1b9fb40e15b43d34 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x1b9fb40e15b43d3b expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.903369:0:6469:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096553.903372:0:6469:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096553.903375:0:6469:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787096553.903419:0:3627:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096553.903431:0:3625:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9c42dc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3625:1873906360246656:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787096553.903466:0:6467:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360246912 00000100:00100000:0.0:1787096553.903469:0:3625:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096553.903472:0:6467:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360246912 00000100:00100000:0.0:1787096553.903500:0:3625:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787096553.903507:0:6467:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360246656 00000100:00100000:1.0:1787096553.903511:0:6467:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360246656 00000100:00100000:0.0:1787096553.903511:0:3625:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d6ea1b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3625:1873906360246784:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787096553.903547:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5 00000100:00100000:0.0:1787096553.903547:0:3625:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096553.903559:0:6467:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d7bf42300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3627:x1873906360246912:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787096553.903569:0:6467:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787096553.903586:0:6467:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cf200/0x1b9fb40e15b43d3b lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x1b9fb40e15b43d34 expref: 12 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096553.903588:0:6465:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360246784 00000100:00100000:3.0:1787096553.903594:0:6465:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360246784 00000100:00100000:0.0:1787096553.903602:0:8378:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6 00010000:00010000:1.0:1787096553.903603:0:6467:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c9c41286000) 00010000:00010000:1.0:1787096553.903609:0:6467:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787096553.903615:0:8378:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42dc9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+12:3625:x1873906360246656:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787096553.903626:0:8378:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787096553.903632:0:6465:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7 00010000:00010000:0.0:1787096553.903642:0:8378:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cfc00/0x1b9fb40e15b43d73 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x1b9fb40e15b43d6c expref: 12 pid: 7409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096553.903643:0:6465:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42dcb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3625:x1873906360246784:12345-0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787096553.903650:0:6467:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d7bf42300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3627:x1873906360246912:12345-0@lo:103:ldlm_bl.0 Request processed in 90us (227us total) trans 0 rc 0/0 00010000:00010000:3.0:1787096553.903656:0:6465:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787096553.903665:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5 00010000:00010000:0.0:1787096553.903669:0:8378:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2:0x0].0x0 (ffff9c9c41286b00) 00010000:00010000:3.0:1787096553.903673:0:6465:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cf800/0x1b9fb40e15b43d57 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x1b9fb40e15b43d50 expref: 11 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096553.903681:0:8378:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787096553.903687:0:6465:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff9c9d48c61500) 00010000:00010000:3.0:1787096553.903702:0:6465:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787096553.903703:0:3627:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9c42dcb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3627:1873906360246912:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787096553.903731:0:8378:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42dc9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3625:x1873906360246656:12345-0@lo:103:ldlm_bl.0 Request processed in 118us (262us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096553.903739:0:8378:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6 00000100:00100000:3.0:1787096553.903743:0:6465:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42dcb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+9:3625:x1873906360246784:12345-0@lo:103:ldlm_bl.0 Request processed in 101us (197us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096553.903751:0:6465:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7 00000100:00100000:0.0:1787096553.903781:0:3625:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d6ea1b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3625:1873906360246784:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787096553.903792:0:3625:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9c42dc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3625:1873906360246656:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787096554.115258:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:3.0:1787096554.115321:0:6475:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352836736 00000100:00100000:3.0:1787096554.115331:0:6475:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352836736 00000100:00100000:3.0:1787096554.115378:0:6475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 327 00000100:00100000:3.0:1787096554.115388:0:6475:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d51325850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:911fcbfa-c40d-448b-aa47-1f7570bacebd+9:2401:x1873906352836736:12345-192.168.203.27@tcp:400:kworker.0 00000100:00100000:3.0:1787096554.115485:0:6475:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d51325850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:911fcbfa-c40d-448b-aa47-1f7570bacebd+9:2401:x1873906352836736:12345-192.168.203.27@tcp:400:kworker.0 Request processed in 98us (236us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096554.115493:0:6475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 327 00010000:00010000:3.0:1787096554.190571:0:7403:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1787096554.465074:0:21972:0:(debug.c:651:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-ec test 23b: truncate EC file to larger size === 19:42:34 (1787096554) 00000100:00100000:3.0:1787096554.529144:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:1.0:1787096554.529203:0:7414:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352836864 00000100:00100000:1.0:1787096554.529211:0:7414:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352836864 00000100:00080000:0.0:1787096554.529256:0:6476:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787096554.529268:0:7414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 163 00000100:00100000:1.0:1787096554.529280:0:7414:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d6ea1b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:47c5a664-3f41-4887-ab67-ea7db94c9f90+5:34300:x1873906352836864:12345-192.168.203.27@tcp:101:lfs.0 00010000:00010000:1.0:1787096554.529301:0:7414:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787096554.529328:0:7414:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cf200/0x1b9fb40e15b43d7a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x8ce29da6e2cc6c32 expref: -99 pid: 7414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.529389:0:7414:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cea00/0x1b9fb40e15b43d81 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7414 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.529429:0:7414:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cea00/0x1b9fb40e15b43d81 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7414 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.529442:0:7414:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cea00/0x1b9fb40e15b43d81 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7414 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787096554.529480:0:7414:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:1.0:1787096554.529484:0:7414:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787096554.529500:0:7414:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cea00/0x1b9fb40e15b43d81 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7414 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.529517:0:7414:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cf200/0x1b9fb40e15b43d7a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c32 expref: 7 pid: 7414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.529528:0:7414:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cea00/0x1b9fb40e15b43d81 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c32 expref: 6 pid: 7414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.529534:0:7414:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c9c41286000) 00010000:00010000:1.0:1787096554.529537:0:7414:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787096554.529541:0:7414:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d6e9cea00, rc 0) 00000100:00100000:1.0:1787096554.529594:0:7414:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d6ea1b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:47c5a664-3f41-4887-ab67-ea7db94c9f90+6:34300:x1873906352836864:12345-192.168.203.27@tcp:101:lfs.0 Request processed in 317us (457us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096554.529600:0:7414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 163 00000100:00100000:0.0:1787096554.530550:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:3.0:1787096554.530580:0:9995:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352836992 00000100:00100000:3.0:1787096554.530586:0:9995:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352836992 00000100:00100000:3.0:1787096554.530619:0:9995:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 164 00000100:00100000:3.0:1787096554.530628:0:9995:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:47c5a664-3f41-4887-ab67-ea7db94c9f90+6:34300:x1873906352836992:12345-192.168.203.27@tcp:101:lfs.0 00010000:00010000:3.0:1787096554.530637:0:9995:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787096554.530647:0:9995:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac3600/0x1b9fb40e15b43d88 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x8ce29da6e2cc6c39 expref: -99 pid: 9995 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530678:0:9995:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac3800/0x1b9fb40e15b43d8f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530688:0:9995:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac3800/0x1b9fb40e15b43d8f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530696:0:9995:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac3800/0x1b9fb40e15b43d8f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530716:0:9995:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac2800/0x1b9fb40e15b43d96 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530724:0:9995:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac2800/0x1b9fb40e15b43d96 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530730:0:9995:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac2800/0x1b9fb40e15b43d96 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530760:0:9995:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac3800/0x1b9fb40e15b43d8f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530766:0:9995:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac3800/0x1b9fb40e15b43d8f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530773:0:9995:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac3800/0x1b9fb40e15b43d8f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530780:0:9995:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac3800/0x1b9fb40e15b43d8f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530785:0:9995:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d6eac3800 already unused, calling callback (ffffffffc19474f0) 00010000:00010000:3.0:1787096554.530789:0:9995:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac3800/0x1b9fb40e15b43d8f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530795:0:9995:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac3800/0x1b9fb40e15b43d8f lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530805:0:9995:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c9c41286000) 00010000:00010000:3.0:1787096554.530807:0:9995:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787096554.530811:0:9995:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac3800/0x1b9fb40e15b43d8f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530822:0:9995:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac3800/0x1b9fb40e15b43d8f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530846:0:9995:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac2800/0x1b9fb40e15b43d96 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530852:0:9995:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac2800/0x1b9fb40e15b43d96 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530858:0:9995:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac2800/0x1b9fb40e15b43d96 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530863:0:9995:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d6eac2800 already unused, calling callback (ffffffffc19474f0) 00010000:00010000:3.0:1787096554.530866:0:9995:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac2800/0x1b9fb40e15b43d96 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530872:0:9995:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac2800/0x1b9fb40e15b43d96 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530879:0:9995:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9280d55e (ffff9c9d6b3f0d00) 00010000:00010000:3.0:1787096554.530881:0:9995:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787096554.530884:0:9995:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac2800/0x1b9fb40e15b43d96 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530890:0:9995:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac2800/0x1b9fb40e15b43d96 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530912:0:9995:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac3600/0x1b9fb40e15b43d88 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c39 expref: 7 pid: 9995 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530932:0:9995:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c9c41286000) 00010000:00010000:3.0:1787096554.530935:0:9995:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787096554.530939:0:9995:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6eac3600/0x1b9fb40e15b43d88 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c39 expref: 7 pid: 9995 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.530947:0:9995:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d6eac3600, rc 0) 00000100:00100000:3.0:1787096554.530978:0:9995:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:47c5a664-3f41-4887-ab67-ea7db94c9f90+6:34300:x1873906352836992:12345-192.168.203.27@tcp:101:lfs.0 Request processed in 352us (432us total) trans 0 rc 301/301 00000100:00100000:3.0:1787096554.530984:0:9995:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 164 00000100:00100000:0.0:1787096554.532898:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:1.0:1787096554.532936:0:6480:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352837248 00000100:00100000:1.0:1787096554.532942:0:6480:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352837248 00000100:00100000:1.0:1787096554.532986:0:6480:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 165 00000100:00080000:0.0:1787096554.532987:0:6476:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787096554.532993:0:6480:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:47c5a664-3f41-4887-ab67-ea7db94c9f90+17:34300:x1873906352837248:12345-192.168.203.27@tcp:36:lfs.0 00000100:00100000:1.0:1787096554.533067:0:6480:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d748ece00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360247040:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787096554.533109:0:6480:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096554.533139:0:7409:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360247040 00000100:00100000:3.0:1787096554.533144:0:7409:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360247040 00000100:00100000:1.0:1787096554.533144:0:6480:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c41c79240 going to sleep for 11 seconds 00000100:00100000:3.0:1787096554.533176:0:7409:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 151 00000100:00100000:3.0:1787096554.533183:0:7409:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d6e994a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6480:x1873906360247040:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787096554.533242:0:7409:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d6e994a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6480:x1873906360247040:12345-0@lo:1000:mdt00_001.0 Request processed in 60us (132us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096554.533249:0:7409:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 151 00000100:00100000:1.0:1787096554.533282:0:6480:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d748ece00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360247040:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787096554.533331:0:6480:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d6e996300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360247168:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787096554.533350:0:6480:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096554.533363:0:6480:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c41c79240 going to sleep for 11 seconds 00000100:00100000:0.0:1787096554.533376:0:6484:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360247168 00000100:00100000:0.0:1787096554.533381:0:6484:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360247168 00000100:00100000:0.0:1787096554.533431:0:6484:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 152 00000100:00100000:0.0:1787096554.533440:0:6484:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d6e994000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6480:x1873906360247168:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787096554.533506:0:6484:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d6e994000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6480:x1873906360247168:12345-0@lo:1000:mdt00_001.0 Request processed in 69us (155us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787096554.533511:0:6484:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 152 00000100:00100000:1.0:1787096554.533531:0:6480:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d6e996300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360247168:0@lo:1000:mdt00_001.0 00010000:00010000:1.0:1787096554.533559:0:6480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.533565:0:6480:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.533571:0:6480:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096554.533577:0:6480:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d6e996300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360247296:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787096554.533594:0:6480:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096554.533613:0:6480:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c41c79d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787096554.533620:0:6485:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360247296 00000100:00100000:0.0:1787096554.533623:0:6485:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360247296 00000100:00100000:0.0:1787096554.533643:0:6485:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 153 00000100:00100000:0.0:1787096554.533646:0:6485:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d748ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6480:x1873906360247296:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787096554.533652:0:6485:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096554.533661:0:6485:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceb200/0x1b9fb40e15b43da4 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x1b9fb40e15b43d9d expref: -99 pid: 6485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.533674:0:6485:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cea00/0x1b9fb40e15b43d81 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c32 expref: 5 pid: 7414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.533689:0:6485:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceb200/0x1b9fb40e15b43da4 lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d9d expref: 9 pid: 6485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.533704:0:6485:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cea00/0x1b9fb40e15b43d81 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c32 expref: 5 pid: 7414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.533712:0:6485:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cea00/0x1b9fb40e15b43d81 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c32 expref: 5 pid: 7414 timeout: 330 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096554.533719:0:6485:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9c42dca680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6485:1873906360247424:192.168.203.27@tcp:104: 00000100:00100000:0.0:1787096554.533753:0:6485:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c42e450c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787096554.534493:0:6485:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9c42dca680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6485:1873906360247424:192.168.203.27@tcp:104: 00010000:00010000:1.0:1787096554.534510:0:6485:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceb200/0x1b9fb40e15b43da4 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d9d expref: 9 pid: 6485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.534517:0:6485:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c9c41286000) 00010000:00010000:1.0:1787096554.534523:0:6485:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787096554.534526:0:6485:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceb200/0x1b9fb40e15b43da4 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d9d expref: 9 pid: 6485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.534531:0:6485:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d7cceb200, rc 0) 00000100:00100000:1.0:1787096554.534561:0:6485:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d748ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6480:x1873906360247296:12345-0@lo:101:mdt00_001.0 Request processed in 914us (967us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096554.534566:0:6485:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 153 00000100:00100000:2.0:1787096554.534605:0:6480:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d6e996300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360247296:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787096554.534619:0:6480:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.534628:0:6480:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096554.534637:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:1.0:1787096554.534667:0:6467:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352837376 00000100:00100000:1.0:1787096554.534671:0:6467:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352837376 00010000:00010000:1.0:1787096554.534692:0:6467:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cea00/0x1b9fb40e15b43d81 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c32 expref: 6 pid: 7414 timeout: 330 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096554.534710:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 65 00000100:00100000:1.0:1787096554.534715:0:6467:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42dcaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:47c5a664-3f41-4887-ab67-ea7db94c9f90+6:2401:x1873906352837376:12345-192.168.203.27@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787096554.534722:0:6467:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787096554.534726:0:6467:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cea00/0x1b9fb40e15b43d81 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c32 expref: 6 pid: 7414 timeout: 330 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.534740:0:6467:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cea00/0x1b9fb40e15b43d81 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c32 expref: 6 pid: 7414 timeout: 330 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.534751:0:6467:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d6e9cea00/0x1b9fb40e15b43d81 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c32 expref: 6 pid: 7414 timeout: 330 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.534760:0:6467:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c9c41286000) 00010000:00010000:1.0:1787096554.534762:0:6467:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceb200/0x1b9fb40e15b43da4 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d9d expref: 8 pid: 6485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.534781:0:6467:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceb200/0x1b9fb40e15b43da4 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x1b9fb40e15b43d9d expref: 8 pid: 6485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.534789:0:6467:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceb200/0x1b9fb40e15b43da4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x1b9fb40e15b43d9d expref: 8 pid: 6485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.534811:0:6467:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceb200/0x1b9fb40e15b43da4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43d9d expref: 8 pid: 6485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096554.534819:0:6467:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d759c2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6467:1873906360247552:0@lo:105: 00000100:00100000:1.0:1787096554.534850:0:6467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096554.534868:0:6467:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c41c79240 going to sleep for 11 seconds 00000100:00100000:0.0:1787096554.534885:0:6463:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360247552 00000100:00100000:0.0:1787096554.534896:0:6463:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360247552 00000100:00100000:0.0:1787096554.534926:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 45 00000100:00100000:0.0:1787096554.534935:0:6463:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d759c0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6467:x1873906360247552:12345-0@lo:105: 00010000:00010000:0.0:1787096554.534944:0:6463:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.534954:0:6463:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.534962:0:6463:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.534969:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.534983:0:6463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.534991:0:6463:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.535010:0:6463:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff9c9d6e9cf400) 00010000:00010000:2.0:1787096554.535031:0:6480:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.535039:0:6480:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096554.535048:0:6463:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d759c0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6467:x1873906360247552:12345-0@lo:105: Request processed in 114us (195us total) trans 0 rc 0/0 00000004:00080000:2.0:1787096554.535051:0:6480:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:0.0:1787096554.535057:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 45 00010000:00010000:2.0:1787096554.535071:0:6480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43dab lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6480 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096554.535072:0:6467:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d759c2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6467:1873906360247552:0@lo:105: 00010000:00010000:2.0:1787096554.535082:0:6480:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43dab lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.535087:0:6480:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43dab lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.535090:0:6467:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787096554.535097:0:6480:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9c42d0bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360247680:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787096554.535125:0:6480:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096554.535127:0:6467:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42dcaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:47c5a664-3f41-4887-ab67-ea7db94c9f90+5:2401:x1873906352837376:12345-192.168.203.27@tcp:103:ldlm_bl.0 Request processed in 407us (491us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096554.535136:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 65 00000100:00100000:2.0:1787096554.535143:0:6480:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9d7bf5e540 going to sleep for 11 seconds 00000100:00100000:0.0:1787096554.535155:0:7723:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360247680 00000100:00100000:0.0:1787096554.535160:0:7723:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360247680 00000100:00100000:0.0:1787096554.535192:0:7723:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 154 00000100:00100000:0.0:1787096554.535199:0:7723:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d6e996300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:6480:x1873906360247680:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787096554.535209:0:7723:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096554.535221:0:7723:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceb000/0x1b9fb40e15b43db2 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x1b9fb40e15b43dab expref: -99 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.535239:0:7723:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceb000/0x1b9fb40e15b43db2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x1b9fb40e15b43dab expref: 10 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.535250:0:7723:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceb000/0x1b9fb40e15b43db2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43dab expref: 10 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.535257:0:7723:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9280d55e (ffff9c9d6b35cc00) 00010000:00010000:0.0:1787096554.535260:0:7723:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096554.535268:0:7723:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d7cceb000, rc 0) 00000100:00100000:0.0:1787096554.535301:0:7723:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d6e996300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:6480:x1873906360247680:12345-0@lo:101:mdt00_001.0 Request processed in 103us (178us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096554.535307:0:7723:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 154 00000100:00100000:2.0:1787096554.535336:0:6480:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9c42d0bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360247680:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787096554.535347:0:6480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43dab lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x1b9fb40e15b43db2 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.535356:0:6480:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43dab lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43db2 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787096554.535369:0:6480:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1787096554.535420:0:6480:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9c42d0bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360247808:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787096554.535445:0:6480:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096554.535459:0:6480:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9d7bf5ea80 going to sleep for 11 seconds 00000100:00100000:3.0:1787096554.535464:0:7409:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360247808 00000100:00100000:3.0:1787096554.535469:0:7409:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360247808 00000100:00100000:3.0:1787096554.535495:0:7409:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 155 00000100:00100000:3.0:1787096554.535501:0:7409:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42cb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6480:x1873906360247808:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787096554.535546:0:7409:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42cb2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6480:x1873906360247808:12345-0@lo:1000:mdt00_001.0 Request processed in 45us (99us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096554.535553:0:7409:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 155 00000100:00100000:2.0:1787096554.535570:0:6480:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9c42d0bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360247808:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787096554.535598:0:6480:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9c42cb3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360247936:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787096554.535619:0:6480:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096554.535635:0:6480:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9d7bf5ea80 going to sleep for 11 seconds 00000100:00100000:1.0:1787096554.535642:0:6484:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360247936 00000100:00100000:1.0:1787096554.535647:0:6484:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360247936 00000100:00100000:1.0:1787096554.535676:0:6484:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 156 00000100:00100000:1.0:1787096554.535683:0:6484:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42cb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6480:x1873906360247936:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787096554.535740:0:6484:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42cb3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6480:x1873906360247936:12345-0@lo:1000:mdt00_001.0 Request processed in 59us (121us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787096554.535745:0:6484:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 156 00000100:00100000:2.0:1787096554.535768:0:6480:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9c42cb3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360247936:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787096554.535839:0:6480:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9c42cb3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360248064:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787096554.535861:0:6480:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096554.535874:0:6480:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9d7bf5ea80 going to sleep for 11 seconds 00000100:00100000:1.0:1787096554.535884:0:6485:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360248064 00000100:00100000:1.0:1787096554.535888:0:6485:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360248064 00000100:00100000:1.0:1787096554.535919:0:6485:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 157 00000100:00100000:1.0:1787096554.535924:0:6485:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42cb1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6480:x1873906360248064:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787096554.535971:0:6485:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42cb1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6480:x1873906360248064:12345-0@lo:1000:mdt00_001.0 Request processed in 47us (109us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787096554.535975:0:6485:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 157 00000100:00100000:2.0:1787096554.535998:0:6480:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9c42cb3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360248064:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787096554.536029:0:6480:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9c42cb3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360248192:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787096554.536045:0:6480:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096554.536057:0:6480:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9d7bf5ea80 going to sleep for 11 seconds 00000100:00100000:0.0:1787096554.536073:0:7723:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360248192 00000100:00100000:0.0:1787096554.536078:0:7723:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360248192 00000100:00100000:0.0:1787096554.536106:0:7723:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 158 00000100:00100000:0.0:1787096554.536113:0:7723:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42d0bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:6480:x1873906360248192:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787096554.536155:0:7723:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42d0bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+10:6480:x1873906360248192:12345-0@lo:1000:mdt00_001.0 Request processed in 43us (109us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096554.536160:0:7723:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 158 00000100:00100000:2.0:1787096554.536173:0:6480:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9c42cb3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360248192:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787096554.536199:0:6480:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9c42cb3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360248320:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787096554.536228:0:6480:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096554.536240:0:6480:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9d7bf5ea80 going to sleep for 11 seconds 00000100:00100000:3.0:1787096554.536249:0:7409:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360248320 00000100:00100000:3.0:1787096554.536253:0:7409:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360248320 00000100:00100000:3.0:1787096554.536278:0:7409:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 159 00000100:00100000:3.0:1787096554.536283:0:7409:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d7bf2a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6480:x1873906360248320:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787096554.536326:0:7409:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d7bf2a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6480:x1873906360248320:12345-0@lo:1000:mdt00_001.0 Request processed in 42us (96us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096554.536330:0:7409:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 159 00000100:00100000:2.0:1787096554.536345:0:6480:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9c42cb3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360248320:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787096554.536370:0:6480:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d7bf28700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360248448:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787096554.536384:0:6480:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096554.536411:0:6480:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9d7bf5ea80 going to sleep for 11 seconds 00000100:00100000:1.0:1787096554.536422:0:6484:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360248448 00000100:00100000:1.0:1787096554.536427:0:6484:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360248448 00000100:00100000:1.0:1787096554.536452:0:6484:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 160 00000100:00100000:1.0:1787096554.536459:0:6484:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d7bf29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6480:x1873906360248448:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787096554.536502:0:6484:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d7bf29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6480:x1873906360248448:12345-0@lo:1000:mdt00_001.0 Request processed in 43us (115us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096554.536508:0:6484:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 160 00000100:00100000:2.0:1787096554.536526:0:6480:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d7bf28700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360248448:0@lo:1000:mdt00_001.0 00000040:00080000:2.0:1787096554.537142:0:6480:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].3, 2152 off38008 00000040:00080000:2.0:1787096554.537193:0:6480:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].3, 2352 off38488 00000004:00080000:2.0:1787096554.537197:0:6480:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff9c9d7bf5e540 with version 6 00000004:00080000:0.0:1787096554.537230:0:7407:0:(osp_trans.c:1330:osp_get_next_request()) ou ffff9c9d589526c0 version 6 rpc_version 6 00000020:00100000:0.0:1787096554.537250:0:7407:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787096554.537255:0:7407:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9c9c42dcb100 x1873906360248704/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787096554.537265:0:7407:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9c42dcb100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7407:1873906360248704:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787096554.537294:0:7407:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096554.537309:0:7407:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c42e45000 going to sleep for 11 seconds 00000100:00100000:1.0:1787096554.537330:0:6485:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360248704 00000100:00100000:1.0:1787096554.537334:0:6485:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360248704 00000100:00100000:1.0:1787096554.537361:0:6485:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 161 00000100:00100000:1.0:1787096554.537367:0:6485:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42dc9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7407:x1873906360248704:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787096554.537857:0:6485:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42dc9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7407:x1873906360248704:12345-0@lo:1000:osp_up0-1.0 Request processed in 489us (560us total) trans 4294967360 rc 0/0 00000100:00100000:1.0:1787096554.537864:0:6485:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 161 00000100:00100000:2.0:1787096554.537905:0:7407:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9c42dcb100 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7407:1873906360248704:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787096554.537977:0:6480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cf200/0x1b9fb40e15b43db9 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.537990:0:6480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cf200/0x1b9fb40e15b43db9 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.538000:0:6480:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cf200/0x1b9fb40e15b43db9 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.538013:0:6480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cee00/0x1b9fb40e15b43dc0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.538026:0:6480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cee00/0x1b9fb40e15b43dc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.538032:0:6480:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cee00/0x1b9fb40e15b43dc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.538050:0:6480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf000/0x1b9fb40e15b43dc7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.538057:0:6480:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf000/0x1b9fb40e15b43dc7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.538065:0:6480:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf000/0x1b9fb40e15b43dc7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096554.538077:0:6480:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d759c2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360248832:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787096554.538099:0:6480:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096554.538116:0:6480:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c41c79240 going to sleep for 11 seconds 00000100:00100000:0.0:1787096554.538126:0:7723:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360248832 00000100:00100000:0.0:1787096554.538130:0:7723:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360248832 00000100:00100000:0.0:1787096554.538149:0:7723:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 162 00000100:00100000:0.0:1787096554.538153:0:7723:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d759c0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+11:6480:x1873906360248832:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787096554.538162:0:7723:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096554.538171:0:7723:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ccea800/0x1b9fb40e15b43dce lrc: 2/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x1b9fb40e15b43dc7 expref: -99 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.538183:0:7723:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ccea800/0x1b9fb40e15b43dce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x1b9fb40e15b43dc7 expref: 12 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.538197:0:7723:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ccea800/0x1b9fb40e15b43dce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43dc7 expref: 12 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.538202:0:7723:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff9c9d6b35c200) 00010000:00010000:0.0:1787096554.538204:0:7723:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096554.538206:0:7723:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d7ccea800, rc 0) 00000100:00100000:0.0:1787096554.538230:0:7723:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d759c0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+12:6480:x1873906360248832:12345-0@lo:101:mdt00_001.0 Request processed in 76us (131us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096554.538234:0:7723:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 162 00000100:00100000:1.0:1787096554.538239:0:6480:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d759c2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6480:1873906360248832:0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787096554.538248:0:6480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf000/0x1b9fb40e15b43dc7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x1b9fb40e15b43dce expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.538255:0:6480:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf000/0x1b9fb40e15b43dc7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43dce expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787096554.538267:0:6480:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000967a6c7f reply state 00000000303f16c4 transno 4294967308 ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cee00/0x1b9fb40e15b43dc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.538274:0:6480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cee00/0x1b9fb40e15b43dc0 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.538283:0:6480:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffff9c9d48c61c00) 00010000:00010000:1.0:1787096554.538285:0:6480:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787096554.538287:0:6480:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787096554.538291:0:6480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf000/0x1b9fb40e15b43dc7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x1b9fb40e15b43dce expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.538296:0:6480:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf000/0x1b9fb40e15b43dc7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x1b9fb40e15b43dce expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787096554.538301:0:6480:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000401:0x5:0x0] 00000004:00010000:1.0:1787096554.538304:0:6480:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000967a6c7f reply state 00000000303f16c4 transno 4294967308 ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cf200/0x1b9fb40e15b43db9 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.538308:0:6480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cf200/0x1b9fb40e15b43db9 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.538311:0:6480:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3:0x0].0x0 (ffff9c9d48c61e00) 00010000:00010000:1.0:1787096554.538313:0:6480:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787096554.538314:0:6480:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:1.0:1787096554.538317:0:6480:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x3:0x0]: returning size 4096 00000004:00080000:1.0:1787096554.538321:0:6480:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787096554.538325:0:6480:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.538329:0:6480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.538333:0:6480:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.538351:0:6480:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787096554.538355:0:6480:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787096554.538357:0:6480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43dab lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x1b9fb40e15b43db2 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.538361:0:6480:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43dab lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x1b9fb40e15b43db2 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.538367:0:6468:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787096554.538372:0:6468:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.538378:0:6468:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d6e9cf400 already unused, calling callback (ffffffffc1953ee0) 00010000:00010000:0.0:1787096554.538382:0:6468:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787096554.538413:0:6468:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.538437:0:6468:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096554.538438:0:6480:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:47c5a664-3f41-4887-ab67-ea7db94c9f90+19:34300:x1873906352837248:12345-192.168.203.27@tcp:36:lfs.0 Request processed in 5444us (5541us total) trans 4294967308 rc 0/0 00010000:00010000:0.0:1787096554.538451:0:6468:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787096554.538455:0:6480:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 165 00010000:00010000:0.0:1787096554.538477:0:6468:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096554.538483:0:3627:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00010000:00010000:0.0:1787096554.538485:0:6468:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf400/0x1b9fb40e15b43d9d lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x1b9fb40e15b43da4 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096554.538495:0:3627:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9c42dcad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3627:1873906360248960:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787096554.538503:0:6468:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096554.538507:0:6468:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096554.538509:0:6468:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787096554.538528:0:3627:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096554.538550:0:8378:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360248960 00000100:00100000:0.0:1787096554.538555:0:8378:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360248960 00000100:00100000:0.0:1787096554.538585:0:8378:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8 00000100:00100000:0.0:1787096554.538592:0:8378:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d759c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+12:3627:x1873906360248960:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787096554.538600:0:8378:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787096554.538610:0:8378:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceb200/0x1b9fb40e15b43da4 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x1b9fb40e15b43d9d expref: 12 pid: 6485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.538618:0:8378:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c9c41286000) 00010000:00010000:0.0:1787096554.538621:0:8378:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787096554.538645:0:8378:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d759c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3627:x1873906360248960:12345-0@lo:103:ldlm_bl.0 Request processed in 54us (117us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096554.538649:0:8378:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8 00000100:00100000:1.0:1787096554.538667:0:3627:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9c42dcad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3627:1873906360248960:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787096554.548953:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:2.0:1787096554.549001:0:6479:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352837504 00000100:00100000:2.0:1787096554.549009:0:6479:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352837504 00000100:00100000:2.0:1787096554.549056:0:6479:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 166 00000100:00100000:2.0:1787096554.549068:0:6479:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42dca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:47c5a664-3f41-4887-ab67-ea7db94c9f90+5:34303:x1873906352837504:12345-192.168.203.27@tcp:101:lfs.0 00010000:00010000:2.0:1787096554.549089:0:6479:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787096554.549104:0:6479:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4b800/0x1b9fb40e15b43dd5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x8ce29da6e2cc6c40 expref: -99 pid: 6479 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.549150:0:6479:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a400/0x1b9fb40e15b43ddc lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6479 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.549161:0:6479:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a400/0x1b9fb40e15b43ddc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6479 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.549169:0:6479:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a400/0x1b9fb40e15b43ddc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6479 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787096554.549205:0:6479:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:2.0:1787096554.549208:0:6479:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1787096554.549221:0:6479:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a400/0x1b9fb40e15b43ddc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6479 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.549237:0:6479:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4b800/0x1b9fb40e15b43dd5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c40 expref: 7 pid: 6479 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.549261:0:6479:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a400/0x1b9fb40e15b43ddc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c40 expref: 6 pid: 6479 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.549267:0:6479:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c9c41286000) 00010000:00010000:2.0:1787096554.549269:0:6479:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787096554.549272:0:6479:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d7ca4a400, rc 0) 00000100:00100000:2.0:1787096554.549311:0:6479:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42dca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:47c5a664-3f41-4887-ab67-ea7db94c9f90+6:34303:x1873906352837504:12345-192.168.203.27@tcp:101:lfs.0 Request processed in 247us (364us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096554.549316:0:6479:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 166 00000100:00100000:3.0:1787096554.550269:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:2.0:1787096554.550300:0:6481:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352837632 00000100:00100000:2.0:1787096554.550303:0:6481:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352837632 00000100:00100000:2.0:1787096554.550331:0:6481:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 167 00000100:00100000:2.0:1787096554.550339:0:6481:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d6ea1b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:47c5a664-3f41-4887-ab67-ea7db94c9f90+6:34303:x1873906352837632:12345-192.168.203.27@tcp:101:lfs.0 00010000:00010000:2.0:1787096554.550348:0:6481:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787096554.550356:0:6481:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a200/0x1b9fb40e15b43de3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x8ce29da6e2cc6c47 expref: -99 pid: 6481 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.550386:0:6481:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43dea lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.550415:0:6481:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43dea lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.550423:0:6481:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43dea lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.550435:0:6481:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43df1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.550450:0:6481:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceb000/0x1b9fb40e15b43db2 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43dab expref: 10 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.550458:0:6481:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43df1 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.550487:0:6481:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceb000/0x1b9fb40e15b43db2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x1b9fb40e15b43dab expref: 10 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096554.550501:0:6481:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d7bf28700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6481:1873906360249088:0@lo:104: 00000100:00100000:2.0:1787096554.550535:0:6481:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096554.550571:0:7402:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360249088 00000100:00100000:0.0:1787096554.550576:0:7402:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360249088 00000100:00100000:2.0:1787096554.550577:0:6481:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9d58952cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787096554.550611:0:7402:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 46 00000100:00100000:0.0:1787096554.550621:0:7402:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d7bf2bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6481:x1873906360249088:12345-0@lo:104: 00010000:00010000:0.0:1787096554.550637:0:7402:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43dab lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x1b9fb40e15b43db2 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.550676:0:7402:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787096554.550681:0:7402:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d7bf2bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6481:x1873906360249088:12345-0@lo:104: Request processed in 64us (146us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096554.550691:0:7402:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 46 00010000:00010000:3.0:1787096554.550706:0:8376:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:2.0:1787096554.550708:0:6481:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d7bf28700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6481:1873906360249088:0@lo:104: 00010000:00010000:3.0:1787096554.550712:0:8376:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43dab lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x1b9fb40e15b43db2 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.550720:0:8376:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d7ca4ae00 already unused, calling callback (ffffffffc1953ee0) 00010000:00010000:3.0:1787096554.550724:0:8376:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43dab lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x1b9fb40e15b43db2 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.550725:0:6481:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43df1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787096554.550744:0:8376:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43dab lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x1b9fb40e15b43db2 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.550765:0:8376:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43dab lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x1b9fb40e15b43db2 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.550769:0:8376:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787096554.550789:0:8376:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43dab lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x1b9fb40e15b43db2 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.550794:0:8376:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43dab lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x1b9fb40e15b43db2 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096554.550806:0:3625:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d6e998000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3625:1873906360249216:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787096554.550807:0:8376:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787096554.550816:0:8376:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787096554.550818:0:8376:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787096554.550842:0:3625:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096554.550866:0:9435:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360249216 00000100:00100000:3.0:1787096554.550871:0:9435:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360249216 00010000:00010000:3.0:1787096554.550891:0:9435:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceb000/0x1b9fb40e15b43db2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x1b9fb40e15b43dab expref: 11 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096554.550915:0:9435:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 66 00000100:00100000:3.0:1787096554.550920:0:9435:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42dcad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+11:3625:x1873906360249216:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787096554.550928:0:9435:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:1.0:1787096554.552253:0:6459:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967360 is committed 00010000:00010000:3.0:1787096554.552312:0:9435:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceb000/0x1b9fb40e15b43db2 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x1b9fb40e15b43dab expref: 9 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.552322:0:9435:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9280d55e (ffff9c9d6b35cc00) 00010000:00010000:3.0:1787096554.552325:0:9435:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43df1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.552330:0:9435:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43df1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.552334:0:9435:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43df1 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096554.552349:0:9435:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787096554.552378:0:9435:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42dcad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:lustre-MDT0001-mdtlov_UUID+8:3625:x1873906360249216:12345-0@lo:103:ldlm_bl.0 Request processed in 1458us (1536us total) trans 0 rc 0/0 00010000:00010000:2.0:1787096554.552379:0:6481:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43df1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096554.552385:0:9435:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 66 00010000:00010000:2.0:1787096554.552420:0:6481:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43df1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096554.552435:0:3625:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d6e998000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3625:1873906360249216:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787096554.552501:0:6481:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d7bf28700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6481:1873906360249344:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787096554.552532:0:6481:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096554.552554:0:6481:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9d58952240 going to sleep for 11 seconds 00000100:00100000:3.0:1787096554.552563:0:7409:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360249344 00000100:00100000:3.0:1787096554.552568:0:7409:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360249344 00000100:00100000:3.0:1787096554.552591:0:7409:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 163 00000100:00100000:3.0:1787096554.552601:0:7409:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d7bf29180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6481:x1873906360249344:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787096554.552662:0:7409:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d7bf29180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6481:x1873906360249344:12345-0@lo:1000:mdt00_002.0 Request processed in 62us (130us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096554.552666:0:7409:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 163 00000100:00100000:2.0:1787096554.552690:0:6481:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d7bf28700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6481:1873906360249344:0@lo:1000:mdt00_002.0 00010000:00010000:2.0:1787096554.552722:0:6481:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a600/0x1b9fb40e15b43df8 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552737:0:6481:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a600/0x1b9fb40e15b43df8 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552745:0:6481:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a600/0x1b9fb40e15b43df8 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787096554.552762:0:6481:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a600/0x1b9fb40e15b43df8 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552772:0:6481:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43dea lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552778:0:6481:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43dea lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552784:0:6481:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43dea lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552789:0:6481:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d7ca4a000 already unused, calling callback (ffffffffc19474f0) 00010000:00010000:2.0:1787096554.552793:0:6481:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43dea lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552799:0:6481:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43dea lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552806:0:6481:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9c9c41286000) 00010000:00010000:2.0:1787096554.552808:0:6481:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096554.552812:0:6481:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43dea lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552817:0:6481:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43dea lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552845:0:6481:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43df1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552851:0:6481:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43df1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552856:0:6481:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43df1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552861:0:6481:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d7ca4b200 already unused, calling callback (ffffffffc19474f0) 00010000:00010000:2.0:1787096554.552864:0:6481:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43df1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552870:0:6481:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43df1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552878:0:6481:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9280d55e (ffff9c9d6b35cc00) 00010000:00010000:2.0:1787096554.552881:0:6481:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096554.552884:0:6481:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43df1 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552889:0:6481:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43df1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552919:0:6481:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a200/0x1b9fb40e15b43de3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c47 expref: 8 pid: 6481 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552931:0:6481:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ca4a600/0x1b9fb40e15b43df8 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c47 expref: 7 pid: 6481 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.552950:0:6481:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3:0x0].0x0 (ffff9c9d6b340c00) 00010000:00010000:2.0:1787096554.552952:0:6481:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787096554.552956:0:6481:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d7ca4a600, rc 0) 00000100:00100000:2.0:1787096554.552993:0:6481:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d6ea1b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:47c5a664-3f41-4887-ab67-ea7db94c9f90+7:34303:x1873906352837632:12345-192.168.203.27@tcp:101:lfs.0 Request processed in 2655us (2725us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096554.553001:0:6481:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 167 00000100:00100000:3.0:1787096554.554180:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:1.0:1787096554.554214:0:18819:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352837760 00000100:00100000:1.0:1787096554.554219:0:18819:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352837760 00000100:00100000:1.0:1787096554.554246:0:18819:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 168 00000100:00100000:1.0:1787096554.554250:0:18819:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d6e998000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:47c5a664-3f41-4887-ab67-ea7db94c9f90+19:34303:x1873906352837760:12345-192.168.203.27@tcp:101:lfs.0 00010000:00010000:1.0:1787096554.554265:0:18819:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787096554.554273:0:18819:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cea00/0x1b9fb40e15b43dff lrc: 2/0,0 mode: --/CR res: [0x240000402:0x3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x8ce29da6e2cc6c4e expref: -99 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.554305:0:18819:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9ce200/0x1b9fb40e15b43e06 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18819 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.554320:0:18819:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9ce200/0x1b9fb40e15b43e06 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.554328:0:18819:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9ce200/0x1b9fb40e15b43e06 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18819 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787096554.554387:0:18819:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x3:0x0]: returning size 4096 00000004:00200000:1.0:1787096554.554410:0:18819:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787096554.554430:0:18819:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cea00/0x1b9fb40e15b43dff lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c4e expref: 21 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.554440:0:18819:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9ce200/0x1b9fb40e15b43e06 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c4e expref: 20 pid: 18819 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.554444:0:18819:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3:0x0].0x0 (ffff9c9d48c61e00) 00010000:00010000:1.0:1787096554.554446:0:18819:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:1.0:1787096554.554447:0:18819:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d6e9ce200, rc 0) 00000100:00100000:1.0:1787096554.554471:0:18819:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d6e998000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:47c5a664-3f41-4887-ab67-ea7db94c9f90+20:34303:x1873906352837760:12345-192.168.203.27@tcp:101:lfs.0 Request processed in 221us (293us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096554.554475:0:18819:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 168 00000100:00100000:0.0:1787096554.555169:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:1.0:1787096554.555203:0:7414:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352837888 00000100:00100000:1.0:1787096554.555209:0:7414:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352837888 00000100:00100000:1.0:1787096554.555239:0:7414:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 169 00000100:00100000:1.0:1787096554.555247:0:7414:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42dca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:47c5a664-3f41-4887-ab67-ea7db94c9f90+20:34303:x1873906352837888:12345-192.168.203.27@tcp:101:lfs.0 00010000:00010000:1.0:1787096554.555254:0:7414:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787096554.555260:0:7414:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cf600/0x1b9fb40e15b43e0d lrc: 2/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x8ce29da6e2cc6c55 expref: -99 pid: 7414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.555288:0:7414:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9ce000/0x1b9fb40e15b43e14 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7414 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.555294:0:7414:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9ce000/0x1b9fb40e15b43e14 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7414 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.555300:0:7414:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9ce000/0x1b9fb40e15b43e14 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7414 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787096554.555320:0:7414:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000400:0x5:0x0]: returning size 4096 00000004:00200000:1.0:1787096554.555323:0:7414:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787096554.555340:0:7414:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9cf600/0x1b9fb40e15b43e0d lrc: 0/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c55 expref: 22 pid: 7414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.555352:0:7414:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9ce000/0x1b9fb40e15b43e14 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c55 expref: 21 pid: 7414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.555360:0:7414:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffff9c9d48c61c00) 00010000:00010000:1.0:1787096554.555362:0:7414:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787096554.555364:0:7414:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d6e9ce000, rc 0) 00000100:00100000:1.0:1787096554.555418:0:7414:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42dca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:47c5a664-3f41-4887-ab67-ea7db94c9f90+21:34303:x1873906352837888:12345-192.168.203.27@tcp:101:lfs.0 Request processed in 148us (225us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096554.555426:0:7414:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 169 00000100:00100000:3.0:1787096554.556092:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:0.0:1787096554.556141:0:9995:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352838016 00000100:00100000:0.0:1787096554.556146:0:9995:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352838016 00000100:00100000:0.0:1787096554.556174:0:9995:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 170 00000100:00100000:0.0:1787096554.556178:0:9995:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d58841180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:47c5a664-3f41-4887-ab67-ea7db94c9f90+7:34303:x1873906352838016:12345-192.168.203.27@tcp:101:lfs.0 00010000:00010000:0.0:1787096554.556185:0:9995:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096554.556193:0:9995:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ccebe00/0x1b9fb40e15b43e1b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x8ce29da6e2cc6c5c expref: -99 pid: 9995 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.556222:0:9995:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43e22 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.556230:0:9995:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ccea800/0x1b9fb40e15b43dce lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x1b9fb40e15b43dc7 expref: 7 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.556235:0:9995:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43e22 lrc: 4/1,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.556251:0:9995:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ccea800/0x1b9fb40e15b43dce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x1b9fb40e15b43dc7 expref: 7 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096554.556261:0:9995:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9c42dcb480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9995:1873906360249472:0@lo:104: 00000100:00100000:0.0:1787096554.556284:0:9995:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096554.556314:0:9995:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9c42e45b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787096554.556321:0:6464:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360249472 00000100:00100000:1.0:1787096554.556326:0:6464:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360249472 00000100:00100000:1.0:1787096554.556356:0:6464:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 47 00000100:00100000:1.0:1787096554.556364:0:6464:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9c42fa5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:9995:x1873906360249472:12345-0@lo:104: 00010000:00010000:1.0:1787096554.556374:0:6464:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf000/0x1b9fb40e15b43dc7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x1b9fb40e15b43dce expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.556447:0:6464:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787096554.556453:0:6464:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9c42fa5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:9995:x1873906360249472:12345-0@lo:104: Request processed in 91us (168us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096554.556458:0:6464:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 47 00010000:00010000:2.0:1787096554.556470:0:8377:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:0.0:1787096554.556472:0:9995:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9c42dcb480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9995:1873906360249472:0@lo:104: 00010000:00010000:2.0:1787096554.556479:0:8377:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf000/0x1b9fb40e15b43dc7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x1b9fb40e15b43dce expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.556488:0:9995:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43e22 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.556503:0:8377:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d6e9cf000 already unused, calling callback (ffffffffc1953ee0) 00010000:00010000:2.0:1787096554.556508:0:8377:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf000/0x1b9fb40e15b43dc7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x1b9fb40e15b43dce expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787096554.556518:0:8377:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf000/0x1b9fb40e15b43dc7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x1b9fb40e15b43dce expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.556543:0:8377:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf000/0x1b9fb40e15b43dc7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x1b9fb40e15b43dce expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.556550:0:8377:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787096554.556571:0:8377:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf000/0x1b9fb40e15b43dc7 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x1b9fb40e15b43dce expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.556579:0:8377:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff9c9d6e9cf000/0x1b9fb40e15b43dc7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x1b9fb40e15b43dce expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096554.556586:0:3626:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d7bf28700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3626:1873906360249600:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787096554.556609:0:8377:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787096554.556612:0:8377:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787096554.556615:0:3626:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787096554.556616:0:8377:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787096554.556656:0:7855:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360249600 00000100:00100000:1.0:1787096554.556660:0:7855:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360249600 00010000:00010000:1.0:1787096554.556676:0:7855:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ccea800/0x1b9fb40e15b43dce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x1b9fb40e15b43dc7 expref: 8 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096554.556701:0:7855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 67 00000100:00100000:1.0:1787096554.556707:0:7855:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d500aac50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3626:x1873906360249600:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787096554.556716:0:7855:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787096554.556978:0:7855:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ccea800/0x1b9fb40e15b43dce lrc: 0/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x1b9fb40e15b43dc7 expref: 8 pid: 7723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.556986:0:7855:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff9c9d6b35c200) 00010000:00010000:1.0:1787096554.556988:0:7855:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43e22 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.556993:0:7855:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43e22 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.556997:0:7855:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43e22 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096554.557009:0:7855:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787096554.557032:0:7855:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d500aac50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+7:3626:x1873906360249600:12345-0@lo:103:ldlm_bl.0 Request processed in 326us (419us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096554.557043:0:7855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 67 00010000:00010000:0.0:1787096554.557048:0:9995:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43e22 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.557057:0:9995:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43e22 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9995 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096554.557078:0:3626:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d7bf28700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3626:1873906360249600:0@lo:103:ldlm_bl.0 00000004:00200000:0.0:1787096554.557087:0:9995:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000401:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787096554.557091:0:9995:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787096554.557115:0:9995:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7ccebe00/0x1b9fb40e15b43e1b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c5c expref: 9 pid: 9995 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.557127:0:9995:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9c9d7cceae00/0x1b9fb40e15b43e22 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c5c expref: 8 pid: 9995 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.557134:0:9995:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff9c9d6b35c200) 00010000:00010000:0.0:1787096554.557137:0:9995:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787096554.557139:0:9995:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d7cceae00, rc 0) 00000100:00100000:0.0:1787096554.557172:0:9995:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d58841180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:47c5a664-3f41-4887-ab67-ea7db94c9f90+8:34303:x1873906352838016:12345-192.168.203.27@tcp:101:lfs.0 Request processed in 992us (1091us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096554.557178:0:9995:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 170 00000100:00100000:3.0:1787096554.557895:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:2.0:1787096554.557928:0:6480:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352838144 00000100:00100000:2.0:1787096554.557934:0:6480:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352838144 00000100:00100000:2.0:1787096554.557960:0:6480:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 171 00000100:00100000:2.0:1787096554.557965:0:6480:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d58840380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:47c5a664-3f41-4887-ab67-ea7db94c9f90+21:34303:x1873906352838144:12345-192.168.203.27@tcp:101:lfs.0 00010000:00010000:2.0:1787096554.557974:0:6480:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787096554.557982:0:6480:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a800/0x1b9fb40e15b43e29 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x8ce29da6e2cc6c63 expref: -99 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.558006:0:6480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ac00/0x1b9fb40e15b43e30 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.558013:0:6480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ac00/0x1b9fb40e15b43e30 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.558018:0:6480:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ac00/0x1b9fb40e15b43e30 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559053:0:6480:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b800/0x1b9fb40e15b43e37 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6480 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559061:0:6480:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b800/0x1b9fb40e15b43e37 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559066:0:6480:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b800/0x1b9fb40e15b43e37 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559113:0:6480:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ac00/0x1b9fb40e15b43e30 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559118:0:6480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ac00/0x1b9fb40e15b43e30 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559122:0:6480:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ac00/0x1b9fb40e15b43e30 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559126:0:6480:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ac00/0x1b9fb40e15b43e30 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559129:0:6480:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d7ca4ac00 already unused, calling callback (ffffffffc19474f0) 00010000:00010000:2.0:1787096554.559132:0:6480:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ac00/0x1b9fb40e15b43e30 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559136:0:6480:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ac00/0x1b9fb40e15b43e30 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559142:0:6480:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffff9c9d48c61c00) 00010000:00010000:2.0:1787096554.559143:0:6480:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096554.559146:0:6480:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ac00/0x1b9fb40e15b43e30 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559149:0:6480:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ac00/0x1b9fb40e15b43e30 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559156:0:6480:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b800/0x1b9fb40e15b43e37 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559159:0:6480:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b800/0x1b9fb40e15b43e37 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559167:0:6480:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b800/0x1b9fb40e15b43e37 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559170:0:6480:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d7ca4b800 already unused, calling callback (ffffffffc19474f0) 00010000:00010000:2.0:1787096554.559171:0:6480:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b800/0x1b9fb40e15b43e37 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559174:0:6480:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b800/0x1b9fb40e15b43e37 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559178:0:6480:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0xae2548a7 (ffff9c9d6b340200) 00010000:00010000:2.0:1787096554.559180:0:6480:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096554.559181:0:6480:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b800/0x1b9fb40e15b43e37 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559184:0:6480:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b800/0x1b9fb40e15b43e37 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6480 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559204:0:6480:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a800/0x1b9fb40e15b43e29 lrc: 1/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c63 expref: 22 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559209:0:6480:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffff9c9d48c61c00) 00010000:00010000:2.0:1787096554.559210:0:6480:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096554.559212:0:6480:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a800/0x1b9fb40e15b43e29 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c63 expref: 22 pid: 6480 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.559216:0:6480:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d7ca4a800, rc 0) 00000100:00100000:2.0:1787096554.559247:0:6480:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d58840380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:47c5a664-3f41-4887-ab67-ea7db94c9f90+21:34303:x1873906352838144:12345-192.168.203.27@tcp:101:lfs.0 Request processed in 1282us (1359us total) trans 0 rc 301/301 00000100:00100000:2.0:1787096554.559258:0:6480:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 171 00000100:00100000:1.0:1787096554.560253:0:3620:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:2.0:1787096554.560285:0:6479:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352838400 00000100:00100000:2.0:1787096554.560290:0:6479:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352838400 00000100:00100000:2.0:1787096554.560317:0:6479:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 172 00000100:00100000:2.0:1787096554.560321:0:6479:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d759c2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:47c5a664-3f41-4887-ab67-ea7db94c9f90+21:34303:x1873906352838400:12345-192.168.203.27@tcp:101:lfs.0 00010000:00010000:2.0:1787096554.560332:0:6479:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787096554.560334:0:6479:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1787096554.560342:0:6479:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d6e9ce000/0x1b9fb40e15b43e14 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c55 expref: 21 pid: 7414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.560349:0:6479:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffff9c9d48c61c00) 00010000:00010000:2.0:1787096554.560351:0:6479:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787096554.560355:0:6479:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b400/0x1b9fb40e15b43e3e lrc: 2/0,0 mode: --/CW res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x8ce29da6e2cc6c6a expref: -99 pid: 6479 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.560434:0:6479:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a200/0x1b9fb40e15b43e45 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6479 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.560441:0:6479:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a200/0x1b9fb40e15b43e45 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.560445:0:6479:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a200/0x1b9fb40e15b43e45 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.560452:0:6479:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43e4c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6479 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.560456:0:6479:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43e4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.560467:0:6479:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43e4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096554.560560:0:6479:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d7bf28700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6479:1873906360249728:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787096554.560593:0:6479:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096554.560611:0:6479:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9d58952240 going to sleep for 11 seconds 00000100:00100000:1.0:1787096554.560630:0:6484:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360249728 00000100:00100000:1.0:1787096554.560635:0:6484:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360249728 00000100:00100000:1.0:1787096554.560664:0:6484:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 164 00000100:00100000:1.0:1787096554.560672:0:6484:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d7bf28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6479:x1873906360249728:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787096554.560731:0:6484:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d7bf28380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6479:x1873906360249728:12345-0@lo:1000:mdt00_000.0 Request processed in 60us (136us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096554.560740:0:6484:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 164 00000100:00100000:2.0:1787096554.560751:0:6479:0:(client.c:3102:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967360 gen 2 00000004:00080000:2.0:1787096554.560756:0:6479:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294967359 committed transno 4294967360 00000004:00080000:2.0:1787096554.560769:0:6479:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294967360 committed transno 4294967360 00000100:00100000:2.0:1787096554.560783:0:6479:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d7bf28700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6479:1873906360249728:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1787096554.561038:0:6479:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43e53 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6479 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.561045:0:6479:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43e53 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.561050:0:6479:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43e53 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787096554.561055:0:6479:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x4:0x0]: returning size 0 00010000:00010000:2.0:1787096554.561069:0:6479:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a200/0x1b9fb40e15b43e45 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.561073:0:6479:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a200/0x1b9fb40e15b43e45 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.561077:0:6479:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a200/0x1b9fb40e15b43e45 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.561079:0:6479:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d7ca4a200 already unused, calling callback (ffffffffc19474f0) 00010000:00010000:2.0:1787096554.561086:0:6479:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a200/0x1b9fb40e15b43e45 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.561090:0:6479:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a200/0x1b9fb40e15b43e45 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.561095:0:6479:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffff9c9d48c61c00) 00010000:00010000:2.0:1787096554.561096:0:6479:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096554.561098:0:6479:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a200/0x1b9fb40e15b43e45 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.561101:0:6479:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a200/0x1b9fb40e15b43e45 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787096554.561107:0:6479:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000966859c0 reply state 000000001c7c86f9 transno 4294967309 ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43e4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.561114:0:6479:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b200/0x1b9fb40e15b43e4c lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6479 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.561118:0:6479:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0xae2548a7 (ffff9c9d6b340a00) 00010000:00010000:2.0:1787096554.561119:0:6479:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787096554.561120:0:6479:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787096554.561135:0:6479:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4b400/0x1b9fb40e15b43e3e lrc: 0/0,0 mode: --/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c6a expref: 23 pid: 6479 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.561142:0:6479:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43e53 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c6a expref: 22 pid: 6479 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.561146:0:6479:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4:0x0].0x0 (ffff9c9d6b340f00) 00010000:00010000:2.0:1787096554.561148:0:6479:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787096554.561154:0:6479:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d7ca4a000, rc 0) 00000100:00100000:2.0:1787096554.561179:0:6479:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d759c2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:47c5a664-3f41-4887-ab67-ea7db94c9f90+23:34303:x1873906352838400:12345-192.168.203.27@tcp:101:lfs.0 Request processed in 858us (929us total) trans 4294967309 rc 0/0 00000100:00100000:2.0:1787096554.561184:0:6479:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 172 00000100:00100000:3.0:1787096554.562225:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:2.0:1787096554.562259:0:6481:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352838656 00000100:00100000:2.0:1787096554.562264:0:6481:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352838656 00000100:00100000:2.0:1787096554.562294:0:6481:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 173 00000100:00100000:2.0:1787096554.562301:0:6481:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d58841f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:47c5a664-3f41-4887-ab67-ea7db94c9f90+23:34303:x1873906352838656:12345-192.168.203.27@tcp:101:lfs.0 00010000:00010000:2.0:1787096554.562314:0:6481:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787096554.562323:0:6481:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ba00/0x1b9fb40e15b43e61 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x8ce29da6e2cc6c71 expref: -99 pid: 6481 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.562371:0:6481:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43e68 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6481 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.562382:0:6481:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43e53 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c6a expref: 24 pid: 6479 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.562389:0:6481:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43e68 lrc: 4/0,1 mode: --/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.562437:0:6481:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43e53 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c6a expref: 24 pid: 6479 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.562449:0:6481:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43e53 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c6a expref: 24 pid: 6479 timeout: 330 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096554.562460:0:6481:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d7bf28700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6481:1873906360249856:192.168.203.27@tcp:104: 00000100:00100000:2.0:1787096554.562519:0:6481:0:(client.c:2692:ptlrpc_set_wait()) set ffff9c9d589520c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787096554.563296:0:6481:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d7bf28700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6481:1873906360249856:192.168.203.27@tcp:104: 00010000:00010000:2.0:1787096554.563308:0:6481:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43e68 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096554.563388:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:0.0:1787096554.563461:0:6465:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352838784 00000100:00100000:0.0:1787096554.563466:0:6465:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352838784 00010000:00010000:0.0:1787096554.563487:0:6465:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43e53 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c6a expref: 25 pid: 6479 timeout: 330 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096554.563519:0:6465:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 68 00000100:00100000:0.0:1787096554.563526:0:6465:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d58840a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:47c5a664-3f41-4887-ab67-ea7db94c9f90+25:2401:x1873906352838784:12345-192.168.203.27@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787096554.563536:0:6465:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787096554.563540:0:6465:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43e53 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c6a expref: 25 pid: 6479 timeout: 330 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.563548:0:6465:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43e53 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c6a expref: 25 pid: 6479 timeout: 330 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.563561:0:6465:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4a000/0x1b9fb40e15b43e53 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c6a expref: 25 pid: 6479 timeout: 330 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.563569:0:6465:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4:0x0].0x0 (ffff9c9d6b340f00) 00010000:00010000:0.0:1787096554.563571:0:6465:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43e68 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.563576:0:6465:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43e68 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.563580:0:6465:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43e68 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096554.563593:0:6465:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787096554.563614:0:6481:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43e68 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096554.563616:0:6465:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d58840a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:47c5a664-3f41-4887-ab67-ea7db94c9f90+24:2401:x1873906352838784:12345-192.168.203.27@tcp:103:ldlm_bl.0 Request processed in 90us (234us total) trans 0 rc 0/0 00010000:00010000:2.0:1787096554.563629:0:6481:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43e68 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096554.563630:0:6465:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 68 00000004:00200000:2.0:1787096554.563642:0:6481:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x4:0x0]: returning size 0 00010000:00010000:2.0:1787096554.563735:0:6481:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43e68 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.563742:0:6481:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43e68 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.563748:0:6481:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43e68 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.563754:0:6481:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9c9d7ca4ae00 already unused, calling callback (ffffffffc19474f0) 00010000:00010000:2.0:1787096554.563758:0:6481:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43e68 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.563764:0:6481:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43e68 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.563773:0:6481:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4:0x0].0x0 (ffff9c9d6b340f00) 00010000:00010000:2.0:1787096554.563776:0:6481:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787096554.563779:0:6481:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43e68 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.563792:0:6481:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ae00/0x1b9fb40e15b43e68 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6481 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.563817:0:6481:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ba00/0x1b9fb40e15b43e61 lrc: 1/0,0 mode: --/CR res: [0x240000402:0x4:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c71 expref: 23 pid: 6481 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.563839:0:6481:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4:0x0].0x0 (ffff9c9d6b340f00) 00010000:00010000:2.0:1787096554.563842:0:6481:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787096554.563846:0:6481:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9c9d7ca4ba00/0x1b9fb40e15b43e61 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x4:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.27@tcp remote: 0x8ce29da6e2cc6c71 expref: 23 pid: 6481 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096554.563860:0:6481:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9c9d7ca4ba00, rc 0) 00000100:00100000:2.0:1787096554.563892:0:6481:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d58841f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:47c5a664-3f41-4887-ab67-ea7db94c9f90+22:34303:x1873906352838656:12345-192.168.203.27@tcp:101:lfs.0 Request processed in 1591us (1668us total) trans 0 rc 301/301 00000100:00100000:2.0:1787096554.563899:0:6481:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 173 00000100:00100000:3.0:1787096554.564875:0:3619:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.27@tcp (source: 12345-192.168.203.27@tcp) 00000100:00100000:1.0:1787096554.564922:0:13479:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906352839040 00000100:00100000:1.0:1787096554.564926:0:13479:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906352839040 00000100:00100000:1.0:1787096554.564952:0:13479:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.27@tcp, seq: 26 00000100:00100000:1.0:1787096554.564957:0:13479:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d58842300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:47c5a664-3f41-4887-ab67-ea7db94c9f90+22:34303:x1873906352839040:12345-192.168.203.27@tcp:35:lfs.0 00000100:00100000:1.0:1787096554.565055:0:13479:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d58842300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:47c5a664-3f41-4887-ab67-ea7db94c9f90+23:34303:x1873906352839040:12345-192.168.203.27@tcp:35:lfs.0 Request processed in 98us (182us total) trans 4294967310 rc 0/0 00000100:00100000:1.0:1787096554.565060:0:13479:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.27@tcp, seq: 26 00000001:02000400:1.0:1787096555.051079:0:22020:0:(debug.c:651:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-ec test_23b: @@@@@@ FAIL: setstripe failed 00000100:00100000:1.0:1787096555.214570:0:3627:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9c9d7bf28700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3627:1873906360249984:0@lo:41:osp-pre-0-1.0 00010000:00010000:0.0:1787096555.214580:0:6469:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787096555.214627:0:3627:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096555.214678:0:6485:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906360249984 00000100:00100000:0.0:1787096555.214686:0:6485:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906360249984 00000100:00100000:0.0:1787096555.214740:0:6485:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 165 00000100:00100000:0.0:1787096555.214750:0:6485:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9c9d7bf42300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3627:x1873906360249984:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1787096555.214781:0:6485:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446219776 avail 1299980288 00000100:00100000:0.0:1787096555.214834:0:6485:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9c9d7bf42300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3627:x1873906360249984:12345-0@lo:41:osp-pre-0-1.0 Request processed in 76us (195us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096555.214841:0:6485:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 165 00000100:00100000:1.0:1787096555.214872:0:3627:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff9c9d7bf28700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3627:1873906360249984:0@lo:41:osp-pre-0-1.0 Debug log: 1330 lines, 1330 kept, 0 dropped, 0 bad.