00010000:00010000:0.0F:1787096580.366768:0:7858:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787096581.098853:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:3.0F:1787096581.098911:0:9989:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345496960 00000100:00100000:3.0:1787096581.098926:0:9989:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345496960 00000100:00080000:0.0:1787096581.098987:0:6484:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787096581.099003:0:9989:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 155 00000100:00100000:3.0:1787096581.099015:0:9989:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff96187c9e5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:70e19547-606a-4706-b016-3476e377d7df+11:34152:x1873906345496960:12345-192.168.201.40@tcp:101:rm.0 00010000:00010000:3.0:1787096581.099034:0:9989:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787096581.099098:0:9989:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff961840030c00/0xa979d75fc166bd16 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe2ded7f867b1fd1 expref: -99 pid: 9989 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.099152:0:9989:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff961840030000/0xa979d75fc166bd1d lrc: 3/1,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9989 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.099162:0:9989:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff961840030000/0xa979d75fc166bd1d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9989 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.099171:0:9989:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff961840030000/0xa979d75fc166bd1d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9989 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787096581.099239:0:9989:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x1:0x0]: returning size 4096 00000004:00200000:3.0:1787096581.099243:0:9989:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:3.0:1787096581.099265:0:9989:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff961840030000/0xa979d75fc166bd1d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9989 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.099283:0:9989:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff961840030c00/0xa979d75fc166bd16 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fd1 expref: 13 pid: 9989 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.099297:0:9989:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff961840030000/0xa979d75fc166bd1d lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fd1 expref: 12 pid: 9989 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.099303:0:9989:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff9618472ca100) 00010000:00010000:3.0:1787096581.099305:0:9989:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787096581.099308:0:9989:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff961840030000, rc 0) 00000100:00100000:3.0:1787096581.099364:0:9989:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff96187c9e5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:70e19547-606a-4706-b016-3476e377d7df+12:34152:x1873906345496960:12345-192.168.201.40@tcp:101:rm.0 Request processed in 352us (517us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096581.099372:0:9989:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 155 00000100:00100000:0.0:1787096581.100751:0:3628:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:3.0:1787096581.100816:0:6490:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345497088 00000100:00100000:3.0:1787096581.100830:0:6490:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345497088 00000100:00100000:3.0:1787096581.100906:0:6490:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 156 00000100:00100000:3.0:1787096581.100917:0:6490:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff96187c9e4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:70e19547-606a-4706-b016-3476e377d7df+12:34152:x1873906345497088:12345-192.168.201.40@tcp:101:rm.0 00010000:00010000:3.0:1787096581.100934:0:6490:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787096581.100951:0:6490:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff961840030e00/0xa979d75fc166bd24 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe2ded7f867b1fd8 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.100998:0:6490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff961840030800/0xa979d75fc166bd2b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6490 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.101010:0:6490:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff961840030800/0xa979d75fc166bd2b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.101018:0:6490:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff961840030800/0xa979d75fc166bd2b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787096581.101052:0:6490:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000400:0x2:0x0]: returning size 4096 00000004:00200000:3.0:1787096581.101056:0:6490:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787096581.101083:0:6490:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff961840030e00/0xa979d75fc166bd24 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fd8 expref: 14 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.101098:0:6490:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff961840030800/0xa979d75fc166bd2b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fd8 expref: 13 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.101105:0:6490:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2:0x0].0x0 (ffff9618472ca500) 00010000:00010000:3.0:1787096581.101119:0:6490:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787096581.101124:0:6490:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff961840030800, rc 0) 00000100:00100000:3.0:1787096581.101199:0:6490:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff96187c9e4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:70e19547-606a-4706-b016-3476e377d7df+13:34152:x1873906345497088:12345-192.168.201.40@tcp:101:rm.0 Request processed in 283us (450us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096581.101207:0:6490:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 156 00000100:00100000:0.0:1787096581.103665:0:3627:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:1.0F:1787096581.103720:0:6489:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345497344 00000100:00100000:1.0:1787096581.103739:0:6489:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345497344 00000100:00100000:1.0:1787096581.103793:0:6489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 157 00000100:00100000:1.0:1787096581.103808:0:6489:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff96187c9e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:70e19547-606a-4706-b016-3476e377d7df+13:34152:x1873906345497344:12345-192.168.201.40@tcp:101:rm.0 00010000:00010000:1.0:1787096581.103826:0:6489:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787096581.103847:0:6489:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96186ddf4a00/0xa979d75fc166bd32 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe2ded7f867b1fdf expref: -99 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787096581.104040:0:6489:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x1:0x0]: returning size 4096 00010000:00010000:1.0:1787096581.104126:0:6489:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff96186ddf4a00/0xa979d75fc166bd32 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fdf expref: 15 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.104139:0:6489:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9618472ca200) 00010000:00010000:1.0:1787096581.104142:0:6489:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787096581.104147:0:6489:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96186ddf4a00/0xa979d75fc166bd32 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fdf expref: 15 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.104163:0:6489:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96186ddf4a00, rc 0) 00000100:00100000:1.0:1787096581.104229:0:6489:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff96187c9e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:70e19547-606a-4706-b016-3476e377d7df+14:34152:x1873906345497344:12345-192.168.201.40@tcp:101:rm.0 Request processed in 428us (576us total) trans 4294967302 rc 301/301 00000100:00100000:1.0:1787096581.104237:0:6489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 157 00000100:00100000:0.0:1787096581.107200:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:1.0:1787096581.107258:0:13534:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345497472 00000100:00100000:1.0:1787096581.107271:0:13534:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345497472 00000100:00100000:1.0:1787096581.107345:0:13534:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 22 00000100:00100000:1.0:1787096581.107358:0:13534:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff96185818d850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:70e19547-606a-4706-b016-3476e377d7df+14:34152:x1873906345497472:12345-192.168.201.40@tcp:37:rm.0 00000100:00100000:0.0:1787096581.110406:0:13534:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff96185818d850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:70e19547-606a-4706-b016-3476e377d7df+14:34152:x1873906345497472:12345-192.168.201.40@tcp:37:rm.0 Request processed in 3043us (3201us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096581.110421:0:13534:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 22 00000100:00100000:2.0F:1787096581.115552:0:3627:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:0.0:1787096581.115605:0:6491:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345497600 00000100:00100000:0.0:1787096581.115615:0:6491:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345497600 00000100:00100000:0.0:1787096581.115672:0:6491:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 23 00000100:00080000:1.0:1787096581.115675:0:6484:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787096581.115683:0:6491:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961848f99880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:70e19547-606a-4706-b016-3476e377d7df+8:34152:x1873906345497600:12345-192.168.201.40@tcp:37:rm.0 00000100:00100000:1.0:1787096581.118510:0:6491:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961848f99880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:70e19547-606a-4706-b016-3476e377d7df+8:34152:x1873906345497600:12345-192.168.201.40@tcp:37:rm.0 Request processed in 2817us (2951us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096581.118536:0:6491:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 23 00000100:00100000:1.0:1787096581.124996:0:3628:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:0.0:1787096581.125138:0:13534:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345497856 00000100:00100000:0.0:1787096581.125150:0:13534:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345497856 00000100:00100000:0.0:1787096581.125239:0:13534:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 24 00000100:00100000:0.0:1787096581.125249:0:13534:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850227b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:70e19547-606a-4706-b016-3476e377d7df+14:34152:x1873906345497856:12345-192.168.201.40@tcp:35:rm.0 00000100:00100000:0.0:1787096581.125466:0:13534:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850227b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:70e19547-606a-4706-b016-3476e377d7df+15:34152:x1873906345497856:12345-192.168.201.40@tcp:35:rm.0 Request processed in 218us (476us total) trans 4294967303 rc 0/0 00000100:00100000:0.0:1787096581.125472:0:13534:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 24 00000100:00100000:2.0:1787096581.128473:0:3627:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:0.0:1787096581.128515:0:6488:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345498112 00000100:00100000:0.0:1787096581.128523:0:6488:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345498112 00000100:00100000:0.0:1787096581.128565:0:6488:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 158 00000100:00100000:0.0:1787096581.128572:0:6488:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961848f98000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:70e19547-606a-4706-b016-3476e377d7df+15:34152:x1873906345498112:12345-192.168.201.40@tcp:101:rm.0 00010000:00010000:0.0:1787096581.128590:0:6488:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096581.128609:0:6488:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fa00/0xa979d75fc166bd40 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe2ded7f867b1fe6 expref: -99 pid: 6488 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.128750:0:6488:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f800/0xa979d75fc166bd47 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.128765:0:6488:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f800/0xa979d75fc166bd47 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.128778:0:6488:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f800/0xa979d75fc166bd47 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787096581.128789:0:6488:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x1:0x0]: returning size 4096 00010000:00010000:0.0:1787096581.128891:0:6488:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fa00/0xa979d75fc166bd40 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fe6 expref: 18 pid: 6488 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.128913:0:6488:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f800/0xa979d75fc166bd47 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x1:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fe6 expref: 17 pid: 6488 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.128923:0:6488:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff9618472ca100) 00010000:00010000:0.0:1787096581.128926:0:6488:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1787096581.128931:0:6488:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96174340f800, rc 0) 00000100:00100000:0.0:1787096581.128976:0:6488:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961848f98000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:70e19547-606a-4706-b016-3476e377d7df+17:34152:x1873906345498112:12345-192.168.201.40@tcp:101:rm.0 Request processed in 404us (534us total) trans 4294967304 rc 0/0 00000100:00100000:0.0:1787096581.128983:0:6488:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 158 00000100:00100000:1.0:1787096581.130614:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:2.0:1787096581.130665:0:6490:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345498240 00000100:00100000:2.0:1787096581.130672:0:6490:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345498240 00000100:00100000:2.0:1787096581.130721:0:6490:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 159 00000100:00100000:2.0:1787096581.130732:0:6490:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850227480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:70e19547-606a-4706-b016-3476e377d7df+8:34152:x1873906345498240:12345-192.168.201.40@tcp:41:rm.0 00000004:00000024:2.0:1787096581.130775:0:6490:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 7847059456 avail 7384014848 00000100:00100000:2.0:1787096581.130837:0:6490:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850227480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:70e19547-606a-4706-b016-3476e377d7df+8:34152:x1873906345498240:12345-192.168.201.40@tcp:41:rm.0 Request processed in 108us (224us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096581.130845:0:6490:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 159 00000100:00100000:1.0:1787096581.132627:0:3628:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:0.0:1787096581.132666:0:6489:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345498368 00000100:00100000:0.0:1787096581.132672:0:6489:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345498368 00000100:00100000:0.0:1787096581.132709:0:6489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 160 00000100:00100000:0.0:1787096581.132719:0:6489:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850225c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:70e19547-606a-4706-b016-3476e377d7df+17:34152:x1873906345498368:12345-192.168.201.40@tcp:101:rm.0 00010000:00010000:0.0:1787096581.132736:0:6489:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096581.132758:0:6489:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e200/0xa979d75fc166bd55 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe2ded7f867b1fed expref: -99 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.132817:0:6489:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ea00/0xa979d75fc166bd5c lrc: 3/1,0 mode: --/CR res: [0x240000400:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6489 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.132832:0:6489:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ea00/0xa979d75fc166bd5c lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.132843:0:6489:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ea00/0xa979d75fc166bd5c lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.132851:0:6489:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ee00/0xa979d75fc166bd63 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6489 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.132856:0:6489:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ee00/0xa979d75fc166bd63 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.132861:0:6489:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ee00/0xa979d75fc166bd63 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133007:0:6489:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e600/0xa979d75fc166bd6a lrc: 3/1,0 mode: --/PR res: [0x240000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6489 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133015:0:6489:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e600/0xa979d75fc166bd6a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x2:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133020:0:6489:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e600/0xa979d75fc166bd6a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x2:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787096581.133052:0:6489:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x2:0x0]: returning size 0 00000004:00200000:0.0:1787096581.133056:0:6489:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787096581.133161:0:6489:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e600/0xa979d75fc166bd6a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x2:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133179:0:6489:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e600/0xa979d75fc166bd6a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133186:0:6489:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ea00/0xa979d75fc166bd5c lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133190:0:6489:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ea00/0xa979d75fc166bd5c lrc: 2/0,0 mode: CR/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133195:0:6489:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ea00/0xa979d75fc166bd5c lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133198:0:6489:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96174340ea00 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:0.0:1787096581.133201:0:6489:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ea00/0xa979d75fc166bd5c lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133206:0:6489:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ea00/0xa979d75fc166bd5c lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133213:0:6489:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2:0x0].0x0 (ffff9618472ca500) 00010000:00010000:0.0:1787096581.133215:0:6489:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096581.133219:0:6489:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ea00/0xa979d75fc166bd5c lrc: 2/0,0 mode: --/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133222:0:6489:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ea00/0xa979d75fc166bd5c lrc: 0/0,0 mode: --/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133229:0:6489:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ee00/0xa979d75fc166bd63 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133233:0:6489:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ee00/0xa979d75fc166bd63 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133241:0:6489:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ee00/0xa979d75fc166bd63 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133244:0:6489:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96174340ee00 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:0.0:1787096581.133245:0:6489:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ee00/0xa979d75fc166bd63 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133248:0:6489:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ee00/0xa979d75fc166bd63 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133253:0:6489:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2:0x0].0x7b711ae6 (ffff9618472cad00) 00010000:00010000:0.0:1787096581.133254:0:6489:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096581.133255:0:6489:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ee00/0xa979d75fc166bd63 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133258:0:6489:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ee00/0xa979d75fc166bd63 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133275:0:6489:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e200/0xa979d75fc166bd55 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fed expref: 19 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133286:0:6489:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e600/0xa979d75fc166bd6a lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fed expref: 18 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.133293:0:6489:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x2:0x0].0x0 (ffff961741937100) 00010000:00010000:0.0:1787096581.133295:0:6489:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787096581.133297:0:6489:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96174340e600, rc 0) 00000100:00100000:0.0:1787096581.133345:0:6489:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850225c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:70e19547-606a-4706-b016-3476e377d7df+18:34152:x1873906345498368:12345-192.168.201.40@tcp:101:rm.0 Request processed in 629us (722us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096581.133357:0:6489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 160 00000100:00100000:1.0:1787096581.134726:0:3627:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:0.0:1787096581.134759:0:6488:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345498624 00000100:00100000:0.0:1787096581.134766:0:6488:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345498624 00000100:00100000:0.0:1787096581.134808:0:6488:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 161 00000100:00100000:0.0:1787096581.134818:0:6488:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850225500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:70e19547-606a-4706-b016-3476e377d7df+18:34152:x1873906345498624:12345-192.168.201.40@tcp:36:rm.0 00010000:00010000:0.0:1787096581.134854:0:6488:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1787096581.134870:0:6488:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e600/0xa979d75fc166bd6a lrc: 0/0,0 mode: --/PR res: [0x240000402:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fed expref: 18 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.134895:0:6488:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x2:0x0].0x0 (ffff961741937100) 00010000:00010000:0.0:1787096581.134909:0:6488:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff961840030800/0xa979d75fc166bd2b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fd8 expref: 17 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.134917:0:6488:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2:0x0].0x0 (ffff9618472ca500) 00010000:00010000:0.0:1787096581.134919:0:6488:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787096581.134929:0:6488:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ec00/0xa979d75fc166bd71 lrc: 3/0,1 mode: --/CW res: [0x240000400:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.134936:0:6488:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ec00/0xa979d75fc166bd71 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.134942:0:6488:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ec00/0xa979d75fc166bd71 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.134949:0:6488:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f000/0xa979d75fc166bd78 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.134956:0:6488:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f000/0xa979d75fc166bd78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.134971:0:6488:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f000/0xa979d75fc166bd78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135026:0:6488:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e800/0xa979d75fc166bd7f lrc: 3/0,1 mode: --/EX res: [0x240000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135034:0:6488:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e800/0xa979d75fc166bd7f lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135041:0:6488:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e800/0xa979d75fc166bd7f lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135213:0:6488:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f400/0xa979d75fc166bd86 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135220:0:6488:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f400/0xa979d75fc166bd86 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135225:0:6488:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f400/0xa979d75fc166bd86 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135230:0:6488:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f400/0xa979d75fc166bd86 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135234:0:6488:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f400/0xa979d75fc166bd86 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135237:0:6488:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f400/0xa979d75fc166bd86 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135251:0:6488:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1787096581.135261:0:6488:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e800/0xa979d75fc166bd7f lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135271:0:6488:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e800/0xa979d75fc166bd7f lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135275:0:6488:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e800/0xa979d75fc166bd7f lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135279:0:6488:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96174340e800 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:1.0:1787096581.135282:0:6476:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787096581.135282:0:6488:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e800/0xa979d75fc166bd7f lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.135292:0:6476:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f400/0xa979d75fc166bd86 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135293:0:6488:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e800/0xa979d75fc166bd7f lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.135310:0:6476:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96174340f400 already unused, calling callback (ffffffffc0bda6f0) 00010000:00010000:1.0:1787096581.135315:0:6476:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f400/0xa979d75fc166bd86 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135315:0:6488:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x2:0x0].0x0 (ffff961741937600) 00010000:00010000:0.0:1787096581.135320:0:6488:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787096581.135324:0:6488:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e800/0xa979d75fc166bd7f lrc: 2/0,0 mode: --/EX res: [0x240000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135331:0:6488:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e800/0xa979d75fc166bd7f lrc: 0/0,0 mode: --/EX res: [0x240000402:0x2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.135342:0:6476:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f400/0xa979d75fc166bd86 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135350:0:6488:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ec00/0xa979d75fc166bd71 lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.135355:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x2:0x0].0x0 (ffff961741937600) 00010000:00010000:0.0:1787096581.135356:0:6488:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ec00/0xa979d75fc166bd71 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.135361:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1787096581.135364:0:6488:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ec00/0xa979d75fc166bd71 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.135366:0:6476:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f400/0xa979d75fc166bd86 lrc: 1/0,0 mode: --/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135369:0:6488:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96174340ec00 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:1.0:1787096581.135374:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f400/0xa979d75fc166bd86 lrc: 0/0,0 mode: --/PW res: [0x240000402:0x2:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135377:0:6488:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ec00/0xa979d75fc166bd71 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135389:0:6488:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ec00/0xa979d75fc166bd71 lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135398:0:6488:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2:0x0].0x0 (ffff9618472ca500) 00010000:00010000:1.0:1787096581.135399:0:6476:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096581.135400:0:6488:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787096581.135404:0:6476:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096581.135404:0:6488:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ec00/0xa979d75fc166bd71 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.135407:0:6476:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096581.135411:0:6488:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ec00/0xa979d75fc166bd71 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787096581.135418:0:6488:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 0000000025ae5a69 reply state 0000000018fc18b6 transno 4294967305 ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f000/0xa979d75fc166bd78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135427:0:6488:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f000/0xa979d75fc166bd78 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x2:0x0].0x7b711ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.135434:0:6488:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2:0x0].0x7b711ae6 (ffff9618472cad00) 00010000:00010000:0.0:1787096581.135436:0:6488:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787096581.135488:0:6488:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787096581.135551:0:6488:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850225500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:70e19547-606a-4706-b016-3476e377d7df+18:34152:x1873906345498624:12345-192.168.201.40@tcp:36:rm.0 Request processed in 733us (826us total) trans 4294967305 rc 0/0 00000100:00100000:0.0:1787096581.135558:0:6488:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 161 00000100:00100000:0.0:1787096581.136916:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:2.0:1787096581.136962:0:6473:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345498752 00000100:00100000:2.0:1787096581.136970:0:6473:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345498752 00000100:00100000:2.0:1787096581.137013:0:6473:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 4 00000100:00100000:2.0:1787096581.137020:0:6473:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff96187c9e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:70e19547-606a-4706-b016-3476e377d7df+8:2400:x1873906345498752:12345-192.168.201.40@tcp:103:rm.0 00010000:00010000:2.0:1787096581.137029:0:6473:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787096581.137042:0:6473:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96174340e400/0xa979d75fc166bc91 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fa0 expref: 8 pid: 9989 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.137054:0:6473:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff961741937700) 00010000:00010000:2.0:1787096581.137061:0:6473:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787096581.137093:0:6473:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff96187c9e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:70e19547-606a-4706-b016-3476e377d7df+7:2400:x1873906345498752:12345-192.168.201.40@tcp:103:rm.0 Request processed in 75us (180us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096581.137098:0:6473:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 4 00000100:00100000:0.0:1787096581.137209:0:3628:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:2.0:1787096581.137300:0:6491:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345499008 00000100:00100000:2.0:1787096581.137304:0:6491:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345499008 00000100:00100000:2.0:1787096581.137338:0:6491:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 25 00000100:00100000:2.0:1787096581.137345:0:6491:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff96187c9e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:70e19547-606a-4706-b016-3476e377d7df+18:34152:x1873906345499008:12345-192.168.201.40@tcp:35:rm.0 00000100:00100000:2.0:1787096581.137499:0:6491:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff96187c9e5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:70e19547-606a-4706-b016-3476e377d7df+19:34152:x1873906345499008:12345-192.168.201.40@tcp:35:rm.0 Request processed in 154us (289us total) trans 4294967306 rc 0/0 00000100:00100000:2.0:1787096581.137508:0:6491:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 25 00000100:00100000:0.0:1787096581.138643:0:3627:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:2.0:1787096581.138675:0:6490:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345499264 00000100:00100000:2.0:1787096581.138679:0:6490:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345499264 00000100:00100000:2.0:1787096581.138713:0:6490:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 162 00000100:00100000:2.0:1787096581.138723:0:6490:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff96187c9e4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:70e19547-606a-4706-b016-3476e377d7df+19:34152:x1873906345499264:12345-192.168.201.40@tcp:36:rm.0 00010000:00010000:2.0:1787096581.138757:0:6490:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1787096581.138774:0:6490:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f800/0xa979d75fc166bd47 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x1:0x0].0x0 bits 0x5/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fe6 expref: 19 pid: 6488 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.138794:0:6490:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff961840030000/0xa979d75fc166bd1d lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fd1 expref: 18 pid: 9989 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.138810:0:6490:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff961845b4b400/0xa979d75fc166bc9f lrc: 0/0,0 mode: --/PR res: [0x240000402:0x1:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fa7 expref: 17 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.138820:0:6490:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff9618472ca100) 00010000:00010000:2.0:1787096581.138824:0:6490:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787096581.138875:0:6490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6490 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.138891:0:6490:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.138901:0:6490:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096581.138931:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961848f98700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353966592:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1787096581.138966:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096581.139004:0:6494:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353966592 00000100:00100000:2.0:1787096581.139006:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff961742a35300 going to sleep for 11 seconds 00000100:00100000:0.0:1787096581.139010:0:6494:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353966592 00000100:00100000:0.0:1787096581.139058:0:6494:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 176 00000100:00100000:0.0:1787096581.139064:0:6494:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961848f9ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:6490:x1873906353966592:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787096581.139075:0:6494:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096581.139087:0:6494:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bd94 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xa979d75fc166bd8d expref: -99 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.139219:0:6494:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9618429d3600/0xa979d75fc166bc75 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1f99 expref: 6 pid: 6488 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.139230:0:6494:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bd94 lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xa979d75fc166bd8d expref: 7 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.139255:0:6494:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9618429d3600/0xa979d75fc166bc75 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1f99 expref: 6 pid: 6488 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.139268:0:6494:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9618429d3600/0xa979d75fc166bc75 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1f99 expref: 6 pid: 6488 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096581.139279:0:6494:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff96187c9e7100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6494:1873906353966720:192.168.201.40@tcp:104: 00000100:00100000:0.0:1787096581.139331:0:6494:0:(client.c:2692:ptlrpc_set_wait()) set ffff9618757cb9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787096581.142298:0:6494:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff96187c9e7100 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6494:1873906353966720:192.168.201.40@tcp:104: 00000100:00100000:1.0:1787096581.142306:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00010000:00010000:2.0:1787096581.142330:0:6494:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bd94 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166bd8d expref: 7 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.142344:0:6494:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9618472ca600) 00010000:00010000:2.0:1787096581.142347:0:6494:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096581.142351:0:6494:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bd94 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166bd8d expref: 7 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.142362:0:6494:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96174340ee00, rc 0) 00000100:00100000:2.0:1787096581.142419:0:6494:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961848f9ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:6490:x1873906353966592:12345-0@lo:101:mdt00_002.0 Request processed in 3354us (3451us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096581.142428:0:6494:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 176 00000100:00100000:1.0:1787096581.142470:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961848f98700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353966592:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787096581.142498:0:6490:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.142511:0:6490:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096581.142521:0:6474:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345499392 00000100:00100000:3.0:1787096581.142529:0:6474:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345499392 00010000:00010000:3.0:1787096581.142558:0:6474:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9618429d3600/0xa979d75fc166bc75 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1f99 expref: 7 pid: 6488 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096581.142603:0:6474:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 59 00000100:00100000:3.0:1787096581.142613:0:6474:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850226a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:70e19547-606a-4706-b016-3476e377d7df+7:2401:x1873906345499392:12345-192.168.201.40@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787096581.142626:0:6474:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787096581.142633:0:6474:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9618429d3600/0xa979d75fc166bc75 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1f99 expref: 7 pid: 6488 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.142645:0:6474:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9618429d3600/0xa979d75fc166bc75 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1f99 expref: 7 pid: 6488 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.142664:0:6474:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9618429d3600/0xa979d75fc166bc75 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1f99 expref: 7 pid: 6488 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.142677:0:6474:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9618472ca600) 00010000:00010000:3.0:1787096581.142682:0:6474:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bd94 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166bd8d expref: 6 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.142692:0:6474:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bd94 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xa979d75fc166bd8d expref: 6 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.142712:0:6474:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bd94 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xa979d75fc166bd8d expref: 6 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.142743:0:6474:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bd94 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166bd8d expref: 6 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096581.142756:0:6474:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961742fd1c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6474:1873906353966848:0@lo:105: 00000100:00100000:3.0:1787096581.142791:0:6474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096581.142821:0:6474:0:(client.c:2692:ptlrpc_set_wait()) set ffff961850131f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787096581.142901:0:7411:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353966848 00000100:00100000:0.0:1787096581.142907:0:7411:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353966848 00000100:00100000:0.0:1787096581.142938:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 39 00000100:00100000:0.0:1787096581.142945:0:7411:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742fd1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6474:x1873906353966848:12345-0@lo:105: 00010000:00010000:0.0:1787096581.142966:0:7411:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.142976:0:7411:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.142992:0:7411:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.143000:0:7411:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.143007:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.143015:0:7411:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.143033:0:7411:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff961845b4b200) 00000100:00100000:0.0:1787096581.143067:0:7411:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742fd1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6474:x1873906353966848:12345-0@lo:105: Request processed in 122us (275us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096581.143073:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 39 00010000:00010000:0.0:1787096581.143097:0:6490:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.143106:0:6490:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787096581.143126:0:6490:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787096581.143158:0:6490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ec00/0xa979d75fc166bd9b lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6490 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.143165:0:6490:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ec00/0xa979d75fc166bd9b lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.143180:0:6490:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ec00/0xa979d75fc166bd9b lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096581.143195:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff96187c9e6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353966976:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787096581.143225:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096581.143248:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff9618757cb240 going to sleep for 11 seconds 00000100:00100000:2.0:1787096581.143263:0:7418:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353966976 00000100:00100000:2.0:1787096581.143268:0:7418:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353966976 00000100:00100000:2.0:1787096581.143304:0:7418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 177 00000100:00100000:2.0:1787096581.143312:0:7418:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff96187c9e5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6490:x1873906353966976:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787096581.143327:0:7418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787096581.143342:0:7418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4a000/0xa979d75fc166bda2 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xa979d75fc166bd9b expref: -99 pid: 7418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.143361:0:7418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4a000/0xa979d75fc166bda2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xa979d75fc166bd9b expref: 8 pid: 7418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096581.143369:0:6474:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961742fd1c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6474:1873906353966848:0@lo:105: 00010000:00010000:3.0:1787096581.143383:0:6474:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787096581.143387:0:7418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4a000/0xa979d75fc166bda2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166bd9b expref: 8 pid: 7418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.143399:0:7418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa5d0027 (ffff961742a27d00) 00010000:00010000:2.0:1787096581.143402:0:7418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096581.143405:0:7418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff961845b4a000, rc 0) 00000100:00100000:3.0:1787096581.143430:0:6474:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850226a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:70e19547-606a-4706-b016-3476e377d7df+6:2401:x1873906345499392:12345-192.168.201.40@tcp:103:ldlm_bl.0 Request processed in 818us (1128us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096581.143464:0:6474:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 59 00000100:00100000:2.0:1787096581.143474:0:7418:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff96187c9e5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6490:x1873906353966976:12345-0@lo:101:mdt00_002.0 Request processed in 162us (247us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096581.143481:0:7418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 177 00000100:00100000:2.0:1787096581.143513:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff96187c9e6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353966976:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787096581.143523:0:6490:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ec00/0xa979d75fc166bd9b lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xa979d75fc166bda2 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.143532:0:6490:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ec00/0xa979d75fc166bd9b lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bda2 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787096581.143544:0:6490:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:2.0:1787096581.143577:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961848f98a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353967104:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787096581.143622:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096581.143643:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff961742a35a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787096581.143929:0:6494:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353967104 00000100:00100000:0.0:1787096581.143934:0:6494:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353967104 00000100:00100000:0.0:1787096581.143965:0:6494:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 178 00000100:00100000:0.0:1787096581.143972:0:6494:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961848f98700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6490:x1873906353967104:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787096581.144042:0:6494:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961848f98700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6490:x1873906353967104:12345-0@lo:1000:mdt00_002.0 Request processed in 71us (420us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096581.144049:0:6494:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 178 00000100:00100000:3.0:1787096581.144263:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961848f98a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353967104:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787096581.144297:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961742fd1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353967232:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787096581.144319:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096581.144338:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff961850131e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787096581.144352:0:7418:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353967232 00000100:00100000:0.0:1787096581.144356:0:7418:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353967232 00000100:00100000:0.0:1787096581.144385:0:7418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 179 00000100:00100000:0.0:1787096581.144391:0:7418:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742fd2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6490:x1873906353967232:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787096581.144490:0:7418:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742fd2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6490:x1873906353967232:12345-0@lo:1000:mdt00_002.0 Request processed in 97us (167us total) trans 0 rc 1/1 00000100:00100000:0.0:1787096581.144496:0:7418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 179 00000100:00100000:3.0:1787096581.144513:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961742fd1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353967232:0@lo:1000:mdt00_002.0 00010000:00010000:3.0:1787096581.144552:0:6490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840030c00/0xa979d75fc166bda9 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6490 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.144561:0:6490:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840030c00/0xa979d75fc166bda9 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.144569:0:6490:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840030c00/0xa979d75fc166bda9 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096581.144589:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961742fd1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353967360:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787096581.144613:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096581.144636:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff961846913180 going to sleep for 11 seconds 00000100:00100000:0.0:1787096581.144688:0:6494:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353967360 00000100:00100000:0.0:1787096581.144692:0:6494:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353967360 00000100:00100000:0.0:1787096581.144725:0:6494:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 180 00000100:00100000:0.0:1787096581.144732:0:6494:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742fd0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6490:x1873906353967360:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787096581.144740:0:6494:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096581.144751:0:6494:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96174340e600/0xa979d75fc166bdb0 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xa979d75fc166bda9 expref: -99 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.144772:0:6494:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96174340e600/0xa979d75fc166bdb0 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xa979d75fc166bda9 expref: 9 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.144783:0:6494:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96174340e600/0xa979d75fc166bdb0 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166bda9 expref: 9 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.144790:0:6494:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff961741937900) 00010000:00010000:0.0:1787096581.144792:0:6494:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787096581.144795:0:6494:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96174340e600, rc 0) 00000100:00100000:0.0:1787096581.144820:0:6494:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742fd0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6490:x1873906353967360:12345-0@lo:101:mdt00_002.0 Request processed in 89us (207us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096581.144826:0:6494:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 180 00000100:00100000:3.0:1787096581.144930:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961742fd1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353967360:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787096581.144941:0:6490:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840030c00/0xa979d75fc166bda9 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xa979d75fc166bdb0 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.145095:0:6490:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840030c00/0xa979d75fc166bda9 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bdb0 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.145119:0:6490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff961840030e00/0xa979d75fc166bdb7 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6490 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.145128:0:6490:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff961840030e00/0xa979d75fc166bdb7 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.145136:0:6490:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff961840030e00/0xa979d75fc166bdb7 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.145152:0:6490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff961840030400/0xa979d75fc166bdbe lrc: 3/0,1 mode: --/EX res: [0x240000400:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6490 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.145160:0:6490:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff961840030400/0xa979d75fc166bdbe lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.145166:0:6490:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff961840030400/0xa979d75fc166bdbe lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.145187:0:6490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6490 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.145200:0:6490:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.145205:0:6490:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096581.145214:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961742fd1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353967488:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787096581.145235:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096581.145252:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff961846913180 going to sleep for 11 seconds 00000100:00100000:0.0:1787096581.145267:0:6494:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353967488 00000100:00100000:0.0:1787096581.145272:0:6494:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353967488 00000100:00100000:0.0:1787096581.145299:0:6494:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 181 00000100:00100000:0.0:1787096581.145329:0:6494:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742fd1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6490:x1873906353967488:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787096581.145338:0:6494:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096581.145346:0:6494:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ea00/0xa979d75fc166bdcc lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xa979d75fc166bdc5 expref: -99 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.145363:0:6494:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4bc00/0xa979d75fc166bcbb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fb5 expref: 5 pid: 9989 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.145372:0:6494:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ea00/0xa979d75fc166bdcc lrc: 4/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xa979d75fc166bdc5 expref: 10 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.145394:0:6494:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4bc00/0xa979d75fc166bcbb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fb5 expref: 5 pid: 9989 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.145425:0:6494:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4bc00/0xa979d75fc166bcbb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fb5 expref: 5 pid: 9989 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096581.145437:0:6494:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff96187c9e6680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6494:1873906353967616:192.168.201.40@tcp:104: 00000100:00100000:0.0:1787096581.145738:0:6494:0:(client.c:2692:ptlrpc_set_wait()) set ffff9618757cb540 going to sleep for 11 seconds 00000100:00100000:1.0:1787096581.150735:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:3.0:1787096581.150828:0:6494:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff96187c9e6680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6494:1873906353967616:192.168.201.40@tcp:104: 00010000:00010000:3.0:1787096581.150859:0:6494:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ea00/0xa979d75fc166bdcc lrc: 3/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166bdc5 expref: 10 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.150873:0:6494:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2:0x0].0x0 (ffff961741937200) 00010000:00010000:3.0:1787096581.150876:0:6494:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787096581.150880:0:6494:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ea00/0xa979d75fc166bdcc lrc: 3/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166bdc5 expref: 10 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.150901:0:6494:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ea00/0xa979d75fc166bdcc lrc: 3/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166bdc5 expref: 10 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.150908:0:6494:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96174340ea00, rc 0) 00000100:00100000:2.0:1787096581.150955:0:6475:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345499520 00000100:00100000:2.0:1787096581.150962:0:6475:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345499520 00000100:00100000:3.0:1787096581.150964:0:6494:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742fd1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6490:x1873906353967488:12345-0@lo:101:mdt00_002.0 Request processed in 5658us (5727us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096581.150973:0:6494:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 181 00010000:00010000:2.0:1787096581.151001:0:6475:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4bc00/0xa979d75fc166bcbb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fb5 expref: 6 pid: 9989 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096581.151036:0:6475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 60 00000100:00100000:2.0:1787096581.151051:0:6475:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850224380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:70e19547-606a-4706-b016-3476e377d7df+6:2400:x1873906345499520:12345-192.168.201.40@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787096581.151064:0:6475:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787096581.151070:0:6475:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4bc00/0xa979d75fc166bcbb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fb5 expref: 6 pid: 9989 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.151082:0:6475:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4bc00/0xa979d75fc166bcbb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fb5 expref: 6 pid: 9989 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.151104:0:6475:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4bc00/0xa979d75fc166bcbb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1fb5 expref: 6 pid: 9989 timeout: 363 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096581.151113:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961742fd1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353967488:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787096581.151117:0:6475:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2:0x0].0x0 (ffff961741937200) 00010000:00010000:2.0:1787096581.151123:0:6475:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ea00/0xa979d75fc166bdcc lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166bdc5 expref: 9 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.151129:0:6490:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.151134:0:6475:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ea00/0xa979d75fc166bdcc lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xa979d75fc166bdc5 expref: 9 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.151146:0:6490:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.151150:0:6475:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ea00/0xa979d75fc166bdcc lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xa979d75fc166bdc5 expref: 9 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.151182:0:6475:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ea00/0xa979d75fc166bdcc lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166bdc5 expref: 9 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096581.151195:0:6475:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961848f98a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6475:1873906353967744:0@lo:105: 00000100:00100000:2.0:1787096581.151233:0:6475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096581.151278:0:8379:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353967744 00000100:00100000:3.0:1787096581.151282:0:8379:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353967744 00000100:00100000:2.0:1787096581.151301:0:6475:0:(client.c:2692:ptlrpc_set_wait()) set ffff961742a35240 going to sleep for 11 seconds 00000100:00100000:3.0:1787096581.151342:0:8379:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 40 00000100:00100000:3.0:1787096581.151347:0:8379:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742d06680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:lustre-MDT0001-mdtlov_UUID+4:6475:x1873906353967744:12345-0@lo:105: 00010000:00010000:3.0:1787096581.151355:0:8379:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.151361:0:8379:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.151367:0:8379:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.151371:0:8379:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.151376:0:8379:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.151381:0:8379:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.151398:0:8379:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff961840031000) 00000100:00100000:3.0:1787096581.151422:0:8379:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742d06680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:lustre-MDT0001-mdtlov_UUID+4:6475:x1873906353967744:12345-0@lo:105: Request processed in 76us (189us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096581.151426:0:8379:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 40 00010000:00010000:2.0:1787096581.151495:0:6490:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.151505:0:6490:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096581.151649:0:6475:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961848f98a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6475:1873906353967744:0@lo:105: 00010000:00010000:3.0:1787096581.151659:0:6475:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787096581.151704:0:6475:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850224380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:70e19547-606a-4706-b016-3476e377d7df+5:2400:x1873906345499520:12345-192.168.201.40@tcp:103:ldlm_bl.0 Request processed in 654us (978us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096581.151719:0:6475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 60 00000100:00100000:2.0:1787096581.152138:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961742d05180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353967872:0@lo:403:mdt00_002.0 00000100:00100000:2.0:1787096581.152172:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096581.152191:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff961742a35240 going to sleep for 11 seconds 00000100:00100000:3.0:1787096581.152217:0:6493:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353967872 00000100:00100000:3.0:1787096581.152221:0:6493:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353967872 00000100:00100000:3.0:1787096581.152254:0:6493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 182 00000100:00100000:3.0:1787096581.152262:0:6493:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742d06d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6490:x1873906353967872:12345-0@lo:403:mdt00_002.0 00000100:00100000:3.0:1787096581.154313:0:6493:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742d06d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6490:x1873906353967872:12345-0@lo:403:mdt00_002.0 Request processed in 2047us (2135us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096581.154383:0:6493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 182 00000100:00100000:1.0:1787096581.154512:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961742d05180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353967872:0@lo:403:mdt00_002.0 00000040:00080000:1.0:1787096581.155270:0:6490:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].2, 936 off35856 00000100:00100000:1.0:1787096581.155475:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961850225180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353968000:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787096581.155520:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096581.155544:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff961741ec3f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787096581.155605:0:6494:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353968000 00000100:00100000:3.0:1787096581.155609:0:6494:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353968000 00000100:00100000:3.0:1787096581.155644:0:6494:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 183 00000100:00100000:3.0:1787096581.155650:0:6494:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850224000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6490:x1873906353968000:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787096581.155709:0:6494:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850224000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6490:x1873906353968000:12345-0@lo:1000:mdt00_002.0 Request processed in 60us (190us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096581.155713:0:6494:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 183 00000100:00100000:1.0:1787096581.159247:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961850225180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353968000:0@lo:1000:mdt00_002.0 00000040:00080000:1.0:1787096581.159299:0:6490:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].2, 1016 off36136 00000004:00080000:1.0:1787096581.159311:0:6490:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff961741ec3780 with version 5 00000004:00080000:2.0:1787096581.159369:0:7416:0:(osp_trans.c:1330:osp_get_next_request()) ou ffff96184550da80 version 5 rpc_version 5 00000020:00100000:2.0:1787096581.159400:0:7416:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787096581.159407:0:7416:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff961742d05180 x1873906353968256/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2592/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787096581.159430:0:7416:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961742d05180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7416:1873906353968256:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787096581.159670:0:7416:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096581.159719:0:6493:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353968256 00000100:00100000:2.0:1787096581.159720:0:7416:0:(client.c:2692:ptlrpc_set_wait()) set ffff961742a350c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787096581.159726:0:6493:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353968256 00000100:00100000:1.0:1787096581.159778:0:6493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:1.0:1787096581.159791:0:6493:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9618581f9050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7416:x1873906353968256:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787096581.160207:0:6493:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9618581f9050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:7416:x1873906353968256:12345-0@lo:1000:osp_up0-1.0 Request processed in 419us (536us total) trans 4294967359 rc 0/0 00000100:00100000:1.0:1787096581.160215:0:6493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000100:00100000:2.0:1787096581.160245:0:7416:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961742d05180 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7416:1873906353968256:0@lo:1000:osp_up0-1.0 00000004:00200000:1.0:1787096581.160531:0:6490:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x1:0x0]: returning size 4096 00010000:00010000:1.0:1787096581.160559:0:6490:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff961840030400/0xa979d75fc166bdbe lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.160568:0:6490:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff961840030400/0xa979d75fc166bdbe lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.160576:0:6490:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff961840030400/0xa979d75fc166bdbe lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.160582:0:6490:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961840030400 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:1.0:1787096581.160586:0:6490:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff961840030400/0xa979d75fc166bdbe lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.160594:0:6490:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff961840030400/0xa979d75fc166bdbe lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.160608:0:6490:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2:0x0].0x0 (ffff9618472ca500) 00010000:00010000:1.0:1787096581.160611:0:6490:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787096581.160616:0:6490:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff961840030400/0xa979d75fc166bdbe lrc: 2/0,0 mode: --/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.160621:0:6490:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff961840030400/0xa979d75fc166bdbe lrc: 0/0,0 mode: --/EX res: [0x240000400:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.160640:0:6490:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.160659:0:6490:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.160665:0:6490:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.160686:0:6490:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787096581.160694:0:6490:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000401:0x2:0x0] 00010000:00010000:1.0:1787096581.160701:0:6490:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff961840030e00/0xa979d75fc166bdb7 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.160707:0:6490:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff961840030e00/0xa979d75fc166bdb7 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.160709:0:10297:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787096581.160717:0:6490:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff961840030e00/0xa979d75fc166bdb7 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.160720:0:10297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.160725:0:6490:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961840030e00 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:1.0:1787096581.160731:0:6490:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff961840030e00/0xa979d75fc166bdb7 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.160735:0:10297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961840031000 already unused, calling callback (ffffffffc1765ee0) 00010000:00010000:1.0:1787096581.160741:0:6490:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff961840030e00/0xa979d75fc166bdb7 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.160743:0:10297:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.160759:0:6490:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff9618472ca100) 00000004:00010000:2.0:1787096581.160760:0:10297:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.160764:0:6490:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787096581.160769:0:6490:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff961840030e00/0xa979d75fc166bdb7 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.160775:0:6490:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff961840030e00/0xa979d75fc166bdb7 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.160795:0:6490:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840030c00/0xa979d75fc166bda9 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bdb0 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.160802:0:6490:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840030c00/0xa979d75fc166bda9 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xa979d75fc166bdb0 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.160806:0:10297:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.160816:0:6490:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840030c00/0xa979d75fc166bda9 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xa979d75fc166bdb0 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.160820:0:10297:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787096581.160857:0:10297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 1/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.160865:0:10297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840031000/0xa979d75fc166bdc5 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xa979d75fc166bdcc expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.160893:0:10297:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787096581.160895:0:10297:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787096581.160898:0:10297:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787096581.161377:0:3634:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961742d04e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3634:1873906353968384:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787096581.161417:0:3634:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096581.161625:0:7412:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353968384 00000100:00100000:0.0:1787096581.161630:0:7412:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353968384 00000100:00100000:0.0:1787096581.161667:0:7412:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5 00000100:00100000:0.0:1787096581.161675:0:7412:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff96187c9e7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+11:3634:x1873906353968384:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787096581.161686:0:7412:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787096581.161703:0:7412:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ea00/0xa979d75fc166bdcc lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xa979d75fc166bdc5 expref: 11 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.161715:0:7412:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2:0x0].0x0 (ffff961741937200) 00010000:00010000:0.0:1787096581.161723:0:7412:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787096581.161757:0:7412:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff96187c9e7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+10:3634:x1873906353968384:12345-0@lo:103:ldlm_bl.0 Request processed in 83us (339us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096581.161764:0:7412:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5 00000100:00100000:0.0:1787096581.161799:0:3634:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961742d04e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3634:1873906353968384:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787096581.162922:0:6490:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787096581.162936:0:6490:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787096581.162939:0:6490:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.162946:0:6490:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.162950:0:6490:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.162957:0:6490:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1787096581.162960:0:6490:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1787096581.162963:0:6490:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ec00/0xa979d75fc166bd9b lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xa979d75fc166bda2 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.162967:0:6490:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ec00/0xa979d75fc166bd9b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5d0027 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xa979d75fc166bda2 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096581.163050:0:6490:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff96187c9e4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:70e19547-606a-4706-b016-3476e377d7df+17:34152:x1873906345499264:12345-192.168.201.40@tcp:36:rm.0 Request processed in 24326us (24407us total) trans 4294967307 rc 0/0 00010000:00010000:0.0:1787096581.163053:0:6477:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787096581.163061:0:6477:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840030c00/0xa979d75fc166bda9 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xa979d75fc166bdb0 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096581.163064:0:6490:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 162 00010000:00010000:0.0:1787096581.163080:0:6477:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961840030c00 already unused, calling callback (ffffffffc1765ee0) 00010000:00010000:0.0:1787096581.163084:0:6477:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840030c00/0xa979d75fc166bda9 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xa979d75fc166bdb0 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787096581.163090:0:6477:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840030c00/0xa979d75fc166bda9 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xa979d75fc166bdb0 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.163110:0:6477:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840030c00/0xa979d75fc166bda9 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xa979d75fc166bdb0 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.163114:0:6477:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787096581.163133:0:6477:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840030c00/0xa979d75fc166bda9 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xa979d75fc166bdb0 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.163138:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff961840030c00/0xa979d75fc166bda9 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xa979d75fc166bdb0 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.163149:0:6477:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787096581.163151:0:6477:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.163154:0:6477:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961845b4b200 already unused, calling callback (ffffffffc1765ee0) 00010000:00010000:0.0:1787096581.163157:0:6477:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096581.163159:0:3635:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff96187c9e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3635:1873906353968512:0@lo:103:ldlm_bl.0 00000004:00010000:0.0:1787096581.163164:0:6477:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.163189:0:6477:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.163193:0:6477:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787096581.163201:0:3635:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787096581.163207:0:6477:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.163212:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b200/0xa979d75fc166bd8d lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xa979d75fc166bd94 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.163220:0:6477:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096581.163223:0:6477:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096581.163224:0:6477:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096581.163231:0:7858:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787096581.163238:0:3636:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff96187c9e4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3636:1873906353968640:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787096581.163259:0:6474:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353968512 00000100:00100000:0.0:1787096581.163263:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096581.163265:0:6474:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353968512 00000100:00100000:0.0:1787096581.163298:0:6473:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353968640 00000100:00100000:0.0:1787096581.163301:0:6473:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353968640 00000100:00100000:1.0:1787096581.163305:0:6474:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6 00000100:00100000:1.0:1787096581.163316:0:6474:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850225880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3635:x1873906353968512:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787096581.163326:0:6474:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1787096581.163326:0:6473:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7 00000100:00100000:0.0:1787096581.163335:0:6473:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff96187c9e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3636:x1873906353968640:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787096581.163339:0:6474:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96174340e600/0xa979d75fc166bdb0 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xa979d75fc166bda9 expref: 11 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.163345:0:6473:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787096581.163348:0:6474:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x1:0x0].0x0 (ffff961741937900) 00010000:00010000:1.0:1787096581.163356:0:6474:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787096581.163359:0:6473:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bd94 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xa979d75fc166bd8d expref: 10 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.163373:0:6473:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9618472ca600) 00010000:00010000:0.0:1787096581.163377:0:6473:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787096581.163392:0:6474:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850225880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+9:3635:x1873906353968512:12345-0@lo:103:ldlm_bl.0 Request processed in 79us (192us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096581.163400:0:6474:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6 00000100:00100000:0.0:1787096581.163413:0:6473:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff96187c9e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3636:x1873906353968640:12345-0@lo:103:ldlm_bl.0 Request processed in 78us (148us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096581.163427:0:6473:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7 00000100:00100000:1.0:1787096581.163460:0:3635:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff96187c9e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3635:1873906353968512:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787096581.163488:0:3636:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff96187c9e4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3636:1873906353968640:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787096581.390830:0:6476:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1787096581.863302:0:21980:0:(debug.c:651:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-ec test 23b: truncate EC file to larger size === 19:43:00 (1787096580) 00000100:00100000:1.0:1787096581.960905:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:0.0:1787096581.960958:0:6489:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345499648 00000100:00100000:0.0:1787096581.960969:0:6489:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345499648 00000100:00100000:0.0:1787096581.961036:0:6489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 163 00000100:00100000:0.0:1787096581.961048:0:6489:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850227800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:70e19547-606a-4706-b016-3476e377d7df+5:34323:x1873906345499648:12345-192.168.201.40@tcp:101:lfs.0 00010000:00010000:0.0:1787096581.961067:0:6489:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096581.961086:0:6489:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fe00/0xa979d75fc166bdd3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe2ded7f867b1ff4 expref: -99 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.961147:0:6489:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fc00/0xa979d75fc166bdda lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6489 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.961162:0:6489:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fc00/0xa979d75fc166bdda lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6489 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.961175:0:6489:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fc00/0xa979d75fc166bdda lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6489 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787096581.961218:0:6489:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:0.0:1787096581.961222:0:6489:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787096581.961241:0:6489:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fc00/0xa979d75fc166bdda lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6489 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.961264:0:6489:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fe00/0xa979d75fc166bdd3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1ff4 expref: 7 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.961278:0:6489:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fc00/0xa979d75fc166bdda lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1ff4 expref: 6 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.961306:0:6489:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9618472ca600) 00010000:00010000:0.0:1787096581.961309:0:6489:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787096581.961314:0:6489:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96174340fc00, rc 0) 00000100:00100000:0.0:1787096581.961377:0:6489:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850227800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:70e19547-606a-4706-b016-3476e377d7df+6:34323:x1873906345499648:12345-192.168.201.40@tcp:101:lfs.0 Request processed in 333us (481us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096581.961384:0:6489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 163 00000100:00100000:2.0:1787096581.962965:0:3628:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:0.0:1787096581.963506:0:6488:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345499776 00000100:00100000:0.0:1787096581.963515:0:6488:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345499776 00000100:00100000:0.0:1787096581.963559:0:6488:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 164 00000100:00100000:0.0:1787096581.963566:0:6488:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742d06d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:70e19547-606a-4706-b016-3476e377d7df+6:34323:x1873906345499776:12345-192.168.201.40@tcp:101:lfs.0 00010000:00010000:0.0:1787096581.963580:0:6488:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096581.963590:0:6488:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ea00/0xa979d75fc166bde1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe2ded7f867b1ffb expref: -99 pid: 6488 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963623:0:6488:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bde8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963631:0:6488:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bde8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963638:0:6488:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bde8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963699:0:6488:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96174340e600/0xa979d75fc166bdef lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963704:0:6488:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96174340e600/0xa979d75fc166bdef lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963718:0:6488:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96174340e600/0xa979d75fc166bdef lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963744:0:6488:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bde8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963748:0:6488:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bde8 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963753:0:6488:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bde8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963756:0:6488:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96174340ee00 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:0.0:1787096581.963759:0:6488:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bde8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963763:0:6488:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bde8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963769:0:6488:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9618472ca600) 00010000:00010000:0.0:1787096581.963771:0:6488:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096581.963774:0:6488:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bde8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963778:0:6488:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ee00/0xa979d75fc166bde8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963784:0:6488:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96174340e600/0xa979d75fc166bdef lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963787:0:6488:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff96174340e600/0xa979d75fc166bdef lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963794:0:6488:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff96174340e600/0xa979d75fc166bdef lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963797:0:6488:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96174340e600 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:0.0:1787096581.963799:0:6488:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96174340e600/0xa979d75fc166bdef lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963802:0:6488:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff96174340e600/0xa979d75fc166bdef lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963806:0:6488:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9280d55e (ffff961741937200) 00010000:00010000:0.0:1787096581.963807:0:6488:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096581.963808:0:6488:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff96174340e600/0xa979d75fc166bdef lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963811:0:6488:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96174340e600/0xa979d75fc166bdef lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963829:0:6488:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ea00/0xa979d75fc166bde1 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1ffb expref: 7 pid: 6488 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963834:0:6488:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9618472ca600) 00010000:00010000:0.0:1787096581.963836:0:6488:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096581.963838:0:6488:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96174340ea00/0xa979d75fc166bde1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1ffb expref: 7 pid: 6488 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.963843:0:6488:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96174340ea00, rc 0) 00000100:00100000:0.0:1787096581.963889:0:6488:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742d06d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:70e19547-606a-4706-b016-3476e377d7df+6:34323:x1873906345499776:12345-192.168.201.40@tcp:101:lfs.0 Request processed in 323us (927us total) trans 0 rc 301/301 00000100:00100000:0.0:1787096581.963902:0:6488:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 164 00000100:00100000:1.0:1787096581.968658:0:3627:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:2.0:1787096581.968753:0:6490:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345500032 00000100:00100000:2.0:1787096581.968763:0:6490:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345500032 00000100:00100000:2.0:1787096581.968824:0:6490:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 165 00000100:00100000:2.0:1787096581.968835:0:6490:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850224a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:70e19547-606a-4706-b016-3476e377d7df+17:34323:x1873906345500032:12345-192.168.201.40@tcp:36:lfs.0 00000100:00100000:2.0:1787096581.968954:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961742d04e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353968768:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787096581.969022:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096581.969059:0:6494:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353968768 00000100:00100000:3.0:1787096581.969065:0:6494:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353968768 00000100:00100000:2.0:1787096581.969067:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff961742a350c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787096581.969102:0:6494:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 185 00000100:00100000:3.0:1787096581.969111:0:6494:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961848f4f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6490:x1873906353968768:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787096581.969187:0:6494:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961848f4f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6490:x1873906353968768:12345-0@lo:1000:mdt00_002.0 Request processed in 78us (163us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096581.969194:0:6494:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 185 00000100:00100000:2.0:1787096581.969270:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961742d04e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353968768:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787096581.969325:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961848f4fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353968896:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787096581.969348:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096581.969370:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff961742a350c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787096581.969427:0:6493:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353968896 00000100:00100000:1.0:1787096581.969431:0:6493:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353968896 00000100:00100000:1.0:1787096581.969525:0:6493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00000100:00100000:1.0:1787096581.969532:0:6493:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961848f4d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6490:x1873906353968896:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787096581.969688:0:6493:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961848f4d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6490:x1873906353968896:12345-0@lo:1000:mdt00_002.0 Request processed in 156us (338us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787096581.969695:0:6493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000100:00100000:2.0:1787096581.969758:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961848f4fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353968896:0@lo:1000:mdt00_002.0 00010000:00010000:2.0:1787096581.969805:0:6490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6490 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.969816:0:6490:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.969824:0:6490:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096581.969835:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961848f4fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353969024:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1787096581.969859:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096581.969894:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff961742a35780 going to sleep for 11 seconds 00000100:00100000:3.0:1787096581.969898:0:6494:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353969024 00000100:00100000:3.0:1787096581.969915:0:6494:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353969024 00000100:00100000:3.0:1787096581.969944:0:6494:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 187 00000100:00100000:3.0:1787096581.969951:0:6494:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742d04e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6490:x1873906353969024:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787096581.969966:0:6494:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787096581.969979:0:6494:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff961840031c00/0xa979d75fc166bdfd lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xa979d75fc166bdf6 expref: -99 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.970002:0:6494:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fc00/0xa979d75fc166bdda lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1ff4 expref: 5 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.970012:0:6494:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff961840031c00/0xa979d75fc166bdfd lrc: 4/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xa979d75fc166bdf6 expref: 9 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.970035:0:6494:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fc00/0xa979d75fc166bdda lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1ff4 expref: 5 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.970048:0:6494:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fc00/0xa979d75fc166bdda lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1ff4 expref: 5 pid: 6489 timeout: 364 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096581.970059:0:6494:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961742fd0000 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6494:1873906353969152:192.168.201.40@tcp:104: 00000100:00100000:3.0:1787096581.970116:0:6494:0:(client.c:2692:ptlrpc_set_wait()) set ffff961846913e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787096581.972026:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:3.0:1787096581.972091:0:6494:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961742fd0000 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6494:1873906353969152:192.168.201.40@tcp:104: 00010000:00010000:3.0:1787096581.972133:0:6494:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff961840031c00/0xa979d75fc166bdfd lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166bdf6 expref: 9 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096581.972163:0:6494:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9618472ca600) 00010000:00010000:3.0:1787096581.972166:0:6494:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787096581.972170:0:6494:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff961840031c00/0xa979d75fc166bdfd lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166bdf6 expref: 9 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096581.972206:0:8380:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345500160 00010000:00010000:3.0:1787096581.972214:0:6494:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff961840031c00, rc 0) 00000100:00100000:0.0:1787096581.972215:0:8380:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345500160 00010000:00010000:0.0:1787096581.972254:0:8380:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fc00/0xa979d75fc166bdda lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1ff4 expref: 6 pid: 6489 timeout: 364 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096581.972303:0:8380:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 61 00000100:00100000:3.0:1787096581.972313:0:6494:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742d04e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6490:x1873906353969024:12345-0@lo:101:mdt00_002.0 Request processed in 2359us (2449us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096581.972323:0:8380:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff9618589c9850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:70e19547-606a-4706-b016-3476e377d7df+6:2403:x1873906345500160:12345-192.168.201.40@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787096581.972332:0:6494:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 187 00010000:00010000:0.0:1787096581.972340:0:8380:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787096581.972348:0:8380:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fc00/0xa979d75fc166bdda lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1ff4 expref: 6 pid: 6489 timeout: 364 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.972365:0:8380:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fc00/0xa979d75fc166bdda lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1ff4 expref: 6 pid: 6489 timeout: 364 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.972384:0:8380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fc00/0xa979d75fc166bdda lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b1ff4 expref: 6 pid: 6489 timeout: 364 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.972397:0:8380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9618472ca600) 00010000:00010000:0.0:1787096581.972403:0:8380:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff961840031c00/0xa979d75fc166bdfd lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166bdf6 expref: 8 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.972412:0:8380:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff961840031c00/0xa979d75fc166bdfd lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xa979d75fc166bdf6 expref: 8 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.972421:0:8380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff961840031c00/0xa979d75fc166bdfd lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0xa979d75fc166bdf6 expref: 8 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.972503:0:8380:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff961840031c00/0xa979d75fc166bdfd lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166bdf6 expref: 8 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096581.972517:0:8380:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff96187c9e4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:8380:1873906353969280:0@lo:105: 00000100:00100000:2.0:1787096581.972544:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961848f4fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353969024:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787096581.972566:0:6490:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096581.972566:0:8380:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787096581.972583:0:6490:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096581.972615:0:8380:0:(client.c:2692:ptlrpc_set_wait()) set ffff9618757cb000 going to sleep for 11 seconds 00000100:00100000:1.0:1787096581.972639:0:7411:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787096581.972652:0:7411:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353969280 00000100:00100000:1.0:1787096581.972656:0:7411:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353969280 00000100:00100000:1.0:1787096581.972690:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 41 00000100:00100000:1.0:1787096581.972698:0:7411:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742ff7450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8380:x1873906353969280:12345-0@lo:105: 00010000:00010000:1.0:1787096581.972710:0:7411:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.972720:0:7411:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.972729:0:7411:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.972738:0:7411:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 5/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.972746:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.972754:0:7411:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 6/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.972802:0:7411:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffff961845b4a600) 00010000:00010000:2.0:1787096581.972831:0:6490:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.972842:0:6490:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096581.972849:0:7411:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742ff7450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8380:x1873906353969280:12345-0@lo:105: Request processed in 149us (280us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096581.972885:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 41 00000100:00100000:0.0:1787096581.972893:0:8380:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff96187c9e4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:8380:1873906353969280:0@lo:105: 00000004:00080000:2.0:1787096581.972902:0:6490:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787096581.972916:0:8380:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787096581.972952:0:6490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b600/0xa979d75fc166be04 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6490 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.972966:0:6490:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b600/0xa979d75fc166be04 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096581.972967:0:8380:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff9618589c9850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:70e19547-606a-4706-b016-3476e377d7df+5:2403:x1873906345500160:12345-192.168.201.40@tcp:103:ldlm_bl.0 Request processed in 642us (931us total) trans 0 rc 0/0 00010000:00010000:2.0:1787096581.972991:0:6490:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b600/0xa979d75fc166be04 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096581.972992:0:8380:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 61 00000100:00100000:2.0:1787096581.973019:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961742d06680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353969408:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1787096581.973068:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096581.973093:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff961742a35780 going to sleep for 11 seconds 00000100:00100000:1.0:1787096581.973109:0:6493:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353969408 00000100:00100000:1.0:1787096581.973115:0:6493:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353969408 00000100:00100000:2.0:1787096581.973164:0:7418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 188 00000100:00100000:2.0:1787096581.973241:0:7418:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961848f4fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6490:x1873906353969408:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787096581.973259:0:7418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787096581.973275:0:7418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4b800/0xa979d75fc166be0b lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xa979d75fc166be04 expref: -99 pid: 7418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.973297:0:7418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4b800/0xa979d75fc166be0b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xa979d75fc166be04 expref: 10 pid: 7418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.973309:0:7418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4b800/0xa979d75fc166be0b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166be04 expref: 10 pid: 7418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.973317:0:7418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9280d55e (ffff961742a27e00) 00010000:00010000:2.0:1787096581.973320:0:7418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096581.973322:0:7418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff961845b4b800, rc 0) 00000100:00100000:2.0:1787096581.973356:0:7418:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961848f4fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6490:x1873906353969408:12345-0@lo:101:mdt00_002.0 Request processed in 183us (288us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096581.973361:0:7418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 188 00000100:00100000:1.0:1787096581.974788:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961742d06680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353969408:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787096581.974806:0:6490:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b600/0xa979d75fc166be04 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xa979d75fc166be0b expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.974819:0:6490:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b600/0xa979d75fc166be04 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166be0b expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787096581.974841:0:6490:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:1.0:1787096581.974900:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961850224e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353969536:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787096581.974943:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096581.974970:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff961842ba7a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787096581.974989:0:6494:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353969536 00000100:00100000:3.0:1787096581.974998:0:6494:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353969536 00000100:00100000:3.0:1787096581.975048:0:6494:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 189 00000100:00100000:3.0:1787096581.975057:0:6494:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850226300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6490:x1873906353969536:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787096581.975132:0:6494:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850226300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6490:x1873906353969536:12345-0@lo:1000:mdt00_002.0 Request processed in 77us (189us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096581.975137:0:6494:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 189 00000100:00100000:1.0:1787096581.975175:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961850224e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353969536:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787096581.975214:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961850224e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353969664:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787096581.975242:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096581.975261:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff961842ba7540 going to sleep for 11 seconds 00000100:00100000:2.0:1787096581.975278:0:7418:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353969664 00000100:00100000:2.0:1787096581.975283:0:7418:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353969664 00000100:00100000:2.0:1787096581.975308:0:7418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 190 00000100:00100000:2.0:1787096581.975331:0:7418:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850224000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6490:x1873906353969664:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787096581.975419:0:7418:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850224000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6490:x1873906353969664:12345-0@lo:1000:mdt00_002.0 Request processed in 91us (176us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787096581.975427:0:7418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 190 00000100:00100000:1.0:1787096581.975509:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961850224e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353969664:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787096581.975586:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961850224e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353969792:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787096581.975614:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096581.975644:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff961842ba7540 going to sleep for 11 seconds 00000100:00100000:3.0:1787096581.975660:0:6494:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353969792 00000100:00100000:3.0:1787096581.975666:0:6494:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353969792 00000100:00100000:3.0:1787096581.975703:0:6494:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 191 00000100:00100000:3.0:1787096581.975718:0:6494:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850225180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6490:x1873906353969792:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787096581.975790:0:6494:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850225180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6490:x1873906353969792:12345-0@lo:1000:mdt00_002.0 Request processed in 80us (174us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787096581.975800:0:6494:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 191 00000100:00100000:1.0:1787096581.975816:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961850224e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353969792:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787096581.975861:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961850224e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353969920:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787096581.975902:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096581.975920:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff961842ba7540 going to sleep for 11 seconds 00000100:00100000:2.0:1787096581.975941:0:7418:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353969920 00000100:00100000:2.0:1787096581.975947:0:7418:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353969920 00000100:00100000:2.0:1787096581.975974:0:7418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:2.0:1787096581.975980:0:7418:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850227100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6490:x1873906353969920:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787096581.976030:0:7418:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850227100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6490:x1873906353969920:12345-0@lo:1000:mdt00_002.0 Request processed in 51us (129us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096581.976035:0:7418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00100000:1.0:1787096581.976060:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961850224e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353969920:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787096581.976099:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961850224e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353970048:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787096581.976127:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096581.976146:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff961842ba7540 going to sleep for 11 seconds 00000100:00100000:3.0:1787096581.976168:0:6494:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353970048 00000100:00100000:3.0:1787096581.976178:0:6494:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353970048 00000100:00100000:3.0:1787096581.976208:0:6494:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 193 00000100:00100000:3.0:1787096581.976218:0:6494:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850226d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6490:x1873906353970048:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787096581.976282:0:6494:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850226d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6490:x1873906353970048:12345-0@lo:1000:mdt00_002.0 Request processed in 67us (154us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096581.976289:0:6494:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 193 00000100:00100000:1.0:1787096581.976319:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961850224e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353970048:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787096581.976356:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961850224e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353970176:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787096581.976382:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096581.976402:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff961842ba7540 going to sleep for 11 seconds 00000100:00100000:2.0:1787096581.976422:0:7418:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353970176 00000100:00100000:2.0:1787096581.976428:0:7418:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353970176 00000100:00100000:2.0:1787096581.976495:0:7418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 194 00000100:00100000:2.0:1787096581.976501:0:7418:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850226a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6490:x1873906353970176:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787096581.976555:0:7418:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850226a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6490:x1873906353970176:12345-0@lo:1000:mdt00_002.0 Request processed in 55us (172us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096581.976559:0:7418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 194 00000100:00100000:0.0:1787096581.976599:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961850224e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353970176:0@lo:1000:mdt00_002.0 00000040:00080000:0.0:1787096581.977368:0:6490:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].3, 2152 off38008 00000040:00080000:0.0:1787096581.977473:0:6490:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].3, 2352 off38488 00000004:00080000:0.0:1787096581.977480:0:6490:0:(osp_trans.c:1582:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffff961846d0c900 with version 6 00000004:00080000:2.0:1787096581.977633:0:7416:0:(osp_trans.c:1330:osp_get_next_request()) ou ffff96184550da80 version 6 rpc_version 6 00000020:00100000:2.0:1787096581.977672:0:7416:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787096581.977682:0:7416:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff961742e3ca80 x1873906353970432/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787096581.977704:0:7416:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961742e3ca80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7416:1873906353970432:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787096581.977801:0:7416:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096581.977825:0:7416:0:(client.c:2692:ptlrpc_set_wait()) set ffff961742a350c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787096581.977837:0:6494:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353970432 00000100:00100000:3.0:1787096581.977844:0:6494:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353970432 00000100:00100000:3.0:1787096581.977883:0:6494:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 195 00000100:00100000:3.0:1787096581.977890:0:6494:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742e3e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7416:x1873906353970432:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787096581.978553:0:6494:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742e3e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7416:x1873906353970432:12345-0@lo:1000:osp_up0-1.0 Request processed in 662us (751us total) trans 4294967360 rc 0/0 00000100:00100000:3.0:1787096581.978575:0:6494:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 195 00000100:00100000:2.0:1787096581.978967:0:7416:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961742e3ca80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7416:1873906353970432:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787096581.979083:0:6490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ea00/0xa979d75fc166be12 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6490 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.979099:0:6490:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ea00/0xa979d75fc166be12 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.979111:0:6490:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ea00/0xa979d75fc166be12 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.979125:0:6490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e600/0xa979d75fc166be19 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6490 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.979129:0:6490:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e600/0xa979d75fc166be19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.979139:0:6490:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e600/0xa979d75fc166be19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.979163:0:6490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ee00/0xa979d75fc166be20 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6490 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.979167:0:6490:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ee00/0xa979d75fc166be20 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.979174:0:6490:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ee00/0xa979d75fc166be20 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096581.979188:0:6490:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff96187c9e5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353970560:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787096581.979219:0:6490:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096581.979241:0:6490:0:(client.c:2692:ptlrpc_set_wait()) set ffff961741ec3300 going to sleep for 11 seconds 00000100:00100000:2.0:1787096581.979251:0:7418:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353970560 00000100:00100000:2.0:1787096581.979256:0:7418:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353970560 00000100:00100000:2.0:1787096581.979286:0:7418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 196 00000100:00100000:2.0:1787096581.979291:0:7418:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff96187c9e7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6490:x1873906353970560:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787096581.979305:0:7418:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787096581.979319:0:7418:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4a800/0xa979d75fc166be27 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xa979d75fc166be20 expref: -99 pid: 7418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.979336:0:7418:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4a800/0xa979d75fc166be27 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0xa979d75fc166be20 expref: 12 pid: 7418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.979345:0:7418:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4a800/0xa979d75fc166be27 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166be20 expref: 12 pid: 7418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.979352:0:7418:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff961742a27200) 00010000:00010000:2.0:1787096581.979355:0:7418:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096581.979357:0:7418:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff961845b4a800, rc 0) 00000100:00100000:2.0:1787096581.979387:0:7418:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff96187c9e7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6490:x1873906353970560:12345-0@lo:101:mdt00_002.0 Request processed in 96us (168us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096581.979391:0:7418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 196 00000100:00100000:0.0:1787096581.979422:0:6490:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff96187c9e5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6490:1873906353970560:0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787096581.979436:0:6490:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ee00/0xa979d75fc166be20 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xa979d75fc166be27 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.979466:0:6490:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ee00/0xa979d75fc166be20 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166be27 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787096581.979488:0:6490:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000d6203ee5 reply state 00000000c3116c6f transno 4294967308 ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e600/0xa979d75fc166be19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.979501:0:6490:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e600/0xa979d75fc166be19 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.979509:0:6490:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffff961741937400) 00010000:00010000:0.0:1787096581.979518:0:6490:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787096581.979521:0:6490:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787096581.979529:0:6490:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ee00/0xa979d75fc166be20 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xa979d75fc166be27 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.979538:0:6490:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ee00/0xa979d75fc166be20 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xa979d75fc166be27 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787096581.979547:0:6490:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000401:0x5:0x0] 00000004:00010000:0.0:1787096581.979552:0:6490:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000d6203ee5 reply state 00000000c3116c6f transno 4294967308 ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ea00/0xa979d75fc166be12 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.979559:0:6490:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340ea00/0xa979d75fc166be12 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.979565:0:6490:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3:0x0].0x0 (ffff961741937200) 00010000:00010000:0.0:1787096581.979568:0:6490:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787096581.979569:0:6490:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787096581.979575:0:6490:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x3:0x0]: returning size 4096 00000004:00080000:0.0:1787096581.979581:0:6490:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787096581.979589:0:6490:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.979597:0:6490:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.979604:0:6490:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.979623:0:6490:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787096581.979627:0:6490:0:(osp_object.c:1397:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787096581.979636:0:6490:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b600/0xa979d75fc166be04 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xa979d75fc166be0b expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.979644:0:6490:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b600/0xa979d75fc166be04 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0xa979d75fc166be0b expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.979647:0:10297:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787096581.979656:0:10297:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.979663:0:10297:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961845b4a600 already unused, calling callback (ffffffffc1765ee0) 00010000:00010000:2.0:1787096581.979668:0:10297:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787096581.979690:0:10297:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.979718:0:10297:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.979726:0:10297:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787096581.979757:0:10297:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096581.979758:0:6490:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850224a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:70e19547-606a-4706-b016-3476e377d7df+19:34323:x1873906345500032:12345-192.168.201.40@tcp:36:lfs.0 Request processed in 10923us (11105us total) trans 4294967308 rc 0/0 00010000:00010000:2.0:1787096581.979774:0:10297:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4a600/0xa979d75fc166bdf6 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0xa979d75fc166bdfd expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096581.979775:0:6490:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 165 00000100:00100000:1.0:1787096581.979785:0:3637:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961742c66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3637:1873906353970688:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787096581.979794:0:10297:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787096581.979797:0:10297:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787096581.979801:0:10297:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787096581.979825:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096581.979990:0:6475:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353970688 00000100:00100000:0.0:1787096581.979994:0:6475:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353970688 00000100:00100000:0.0:1787096581.980026:0:6475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8 00000100:00100000:0.0:1787096581.980033:0:6475:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850224380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3637:x1873906353970688:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787096581.980042:0:6475:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787096581.980064:0:6475:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff961840031c00/0xa979d75fc166bdfd lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xa979d75fc166bdf6 expref: 12 pid: 6494 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.980077:0:6475:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9618472ca600) 00010000:00010000:0.0:1787096581.980081:0:6475:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787096581.980118:0:6475:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850224380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3637:x1873906353970688:12345-0@lo:103:ldlm_bl.0 Request processed in 78us (286us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096581.980125:0:6475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8 00000100:00100000:1.0:1787096581.980153:0:3637:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961742c66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3637:1873906353970688:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787096581.992607:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:0.0:1787096581.992674:0:6489:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345500288 00000100:00100000:0.0:1787096581.992687:0:6489:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345500288 00000100:00100000:0.0:1787096581.992759:0:6489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 166 00000100:00100000:0.0:1787096581.992772:0:6489:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850225880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:70e19547-606a-4706-b016-3476e377d7df+5:34328:x1873906345500288:12345-192.168.201.40@tcp:101:lfs.0 00010000:00010000:0.0:1787096581.992800:0:6489:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096581.992838:0:6489:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fc00/0xa979d75fc166be2e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe2ded7f867b2002 expref: -99 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.992941:0:6489:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fe00/0xa979d75fc166be35 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6489 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.992959:0:6489:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fe00/0xa979d75fc166be35 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6489 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.992973:0:6489:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fe00/0xa979d75fc166be35 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6489 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787096581.993031:0:6489:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:0.0:1787096581.993035:0:6489:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787096581.993058:0:6489:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fe00/0xa979d75fc166be35 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6489 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.993090:0:6489:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fc00/0xa979d75fc166be2e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b2002 expref: 7 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.993110:0:6489:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff96174340fe00/0xa979d75fc166be35 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b2002 expref: 6 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.993119:0:6489:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9618472ca600) 00010000:00010000:0.0:1787096581.993123:0:6489:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787096581.993128:0:6489:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96174340fe00, rc 0) 00000100:00100000:0.0:1787096581.993218:0:6489:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850225880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:70e19547-606a-4706-b016-3476e377d7df+6:34328:x1873906345500288:12345-192.168.201.40@tcp:101:lfs.0 Request processed in 448us (620us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096581.993226:0:6489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 166 00000100:00100000:3.0:1787096581.995209:0:3628:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:2.0:1787096581.995279:0:6488:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345500416 00000100:00100000:2.0:1787096581.995289:0:6488:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345500416 00000100:00100000:2.0:1787096581.995348:0:6488:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 167 00000100:00100000:2.0:1787096581.995359:0:6488:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742fd0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:70e19547-606a-4706-b016-3476e377d7df+6:34328:x1873906345500416:12345-192.168.201.40@tcp:101:lfs.0 00010000:00010000:2.0:1787096581.995380:0:6488:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787096581.995395:0:6488:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4a600/0xa979d75fc166be3c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe2ded7f867b2009 expref: -99 pid: 6488 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.995484:0:6488:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4ae00/0xa979d75fc166be43 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.995496:0:6488:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4ae00/0xa979d75fc166be43 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.995515:0:6488:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4ae00/0xa979d75fc166be43 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.995524:0:6488:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be4a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.995532:0:6488:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4b800/0xa979d75fc166be0b lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166be04 expref: 10 pid: 7418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.995538:0:6488:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be4a lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.995568:0:6488:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4b800/0xa979d75fc166be0b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xa979d75fc166be04 expref: 10 pid: 7418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096581.995582:0:6488:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961742c64700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6488:1873906353970816:0@lo:104: 00000100:00100000:2.0:1787096581.995622:0:6488:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096581.995667:0:6472:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353970816 00000100:00100000:1.0:1787096581.995672:0:6472:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353970816 00000100:00100000:2.0:1787096581.995688:0:6488:0:(client.c:2692:ptlrpc_set_wait()) set ffff961742a350c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787096581.995711:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 42 00000100:00100000:1.0:1787096581.995719:0:6472:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742c65880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6488:x1873906353970816:12345-0@lo:104: 00010000:00010000:1.0:1787096581.995734:0:6472:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b600/0xa979d75fc166be04 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xa979d75fc166be0b expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787096581.995793:0:6472:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787096581.995806:0:6472:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742c65880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6488:x1873906353970816:12345-0@lo:104: Request processed in 88us (181us total) trans 0 rc 0/0 00000100:00100000:1.0:1787096581.995814:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 42 00010000:00010000:0.0:1787096581.995819:0:6477:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787096581.995836:0:6477:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b600/0xa979d75fc166be04 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xa979d75fc166be0b expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096581.995839:0:6488:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961742c64700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6488:1873906353970816:0@lo:104: 00010000:00010000:0.0:1787096581.995851:0:6477:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961845b4b600 already unused, calling callback (ffffffffc1765ee0) 00010000:00010000:0.0:1787096581.995858:0:6477:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b600/0xa979d75fc166be04 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xa979d75fc166be0b expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.995867:0:6488:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be4a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787096581.995879:0:6477:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b600/0xa979d75fc166be04 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xa979d75fc166be0b expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.995929:0:6477:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b600/0xa979d75fc166be04 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xa979d75fc166be0b expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.995936:0:6477:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787096581.995967:0:6477:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b600/0xa979d75fc166be04 lrc: 1/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xa979d75fc166be0b expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.995975:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff961845b4b600/0xa979d75fc166be04 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xa979d75fc166be0b expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096581.995991:0:3634:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff96187c9e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3634:1873906353970944:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787096581.995998:0:6477:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096581.996001:0:6477:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096581.996004:0:6477:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787096581.996017:0:3634:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787096581.996051:0:7412:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353970944 00000100:00100000:0.0:1787096581.996056:0:7412:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353970944 00010000:00010000:0.0:1787096581.996080:0:7412:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4b800/0xa979d75fc166be0b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xa979d75fc166be04 expref: 11 pid: 7418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096581.996111:0:7412:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 62 00000100:00100000:0.0:1787096581.996117:0:7412:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742c64700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+11:3634:x1873906353970944:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787096581.996148:0:7412:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:2.0:1787096581.997951:0:6467:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967360 is committed 00010000:00010000:0.0:1787096581.998041:0:7412:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4b800/0xa979d75fc166be0b lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xa979d75fc166be04 expref: 9 pid: 7418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.998058:0:7412:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9280d55e (ffff961742a27e00) 00010000:00010000:0.0:1787096581.998064:0:7412:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be4a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.998073:0:7412:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be4a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.998081:0:7412:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be4a lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096581.998112:0:7412:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787096581.998133:0:6488:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be4a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998164:0:6488:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be4a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096581.998183:0:7412:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742c64700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3634:x1873906353970944:12345-0@lo:103:ldlm_bl.0 Request processed in 2063us (2162us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096581.998193:0:7412:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 62 00000100:00100000:3.0:1787096581.998239:0:3634:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff96187c9e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3634:1873906353970944:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787096581.998280:0:6488:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961742c66680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6488:1873906353971072:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787096581.998322:0:6488:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096581.998360:0:6488:0:(client.c:2692:ptlrpc_set_wait()) set ffff961742a356c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787096581.998432:0:6494:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353971072 00000100:00100000:3.0:1787096581.998437:0:6494:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353971072 00000100:00100000:3.0:1787096581.998506:0:6494:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 197 00000100:00100000:3.0:1787096581.998516:0:6494:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742ccb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6488:x1873906353971072:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787096581.998599:0:6494:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742ccb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6488:x1873906353971072:12345-0@lo:1000:mdt00_000.0 Request processed in 84us (277us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096581.998604:0:6494:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 197 00000100:00100000:2.0:1787096581.998641:0:6488:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961742c66680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6488:1873906353971072:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1787096581.998690:0:6488:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4aa00/0xa979d75fc166be51 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998703:0:6488:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4aa00/0xa979d75fc166be51 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998713:0:6488:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4aa00/0xa979d75fc166be51 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787096581.998736:0:6488:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4aa00/0xa979d75fc166be51 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998749:0:6488:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4ae00/0xa979d75fc166be43 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998756:0:6488:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4ae00/0xa979d75fc166be43 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998763:0:6488:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4ae00/0xa979d75fc166be43 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998769:0:6488:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961845b4ae00 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:2.0:1787096581.998774:0:6488:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4ae00/0xa979d75fc166be43 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998788:0:6488:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4ae00/0xa979d75fc166be43 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998798:0:6488:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9618472ca600) 00010000:00010000:2.0:1787096581.998801:0:6488:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096581.998805:0:6488:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4ae00/0xa979d75fc166be43 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998811:0:6488:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4ae00/0xa979d75fc166be43 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998821:0:6488:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be4a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998827:0:6488:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be4a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998833:0:6488:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be4a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998838:0:6488:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961845b4be00 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:2.0:1787096581.998841:0:6488:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be4a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998846:0:6488:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be4a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998854:0:6488:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9280d55e (ffff961742a27e00) 00010000:00010000:2.0:1787096581.998857:0:6488:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787096581.998859:0:6488:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be4a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998889:0:6488:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be4a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9280d55e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998929:0:6488:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4a600/0xa979d75fc166be3c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b2009 expref: 8 pid: 6488 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998944:0:6488:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4aa00/0xa979d75fc166be51 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b2009 expref: 7 pid: 6488 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096581.998951:0:6488:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3:0x0].0x0 (ffff961742a26600) 00010000:00010000:2.0:1787096581.998954:0:6488:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787096581.998956:0:6488:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff961845b4aa00, rc 0) 00000100:00100000:2.0:1787096581.999017:0:6488:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742fd0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:70e19547-606a-4706-b016-3476e377d7df+7:34328:x1873906345500416:12345-192.168.201.40@tcp:101:lfs.0 Request processed in 3660us (3817us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096581.999025:0:6488:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 167 00000100:00100000:1.0:1787096582.002159:0:3627:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:0.0:1787096582.002207:0:6490:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345500544 00000100:00100000:0.0:1787096582.002217:0:6490:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345500544 00000100:00100000:0.0:1787096582.002266:0:6490:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 168 00000100:00100000:0.0:1787096582.002275:0:6490:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850224700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:70e19547-606a-4706-b016-3476e377d7df+19:34328:x1873906345500544:12345-192.168.201.40@tcp:101:lfs.0 00010000:00010000:0.0:1787096582.002300:0:6490:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096582.002318:0:6490:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e000/0xa979d75fc166be58 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe2ded7f867b2010 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.002383:0:6490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e800/0xa979d75fc166be5f lrc: 3/1,0 mode: --/PR res: [0x240000402:0x3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6490 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.002418:0:6490:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e800/0xa979d75fc166be5f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.002430:0:6490:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e800/0xa979d75fc166be5f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x10/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787096582.002589:0:6490:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x3:0x0]: returning size 4096 00000004:00200000:0.0:1787096582.002596:0:6490:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787096582.002636:0:6490:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e000/0xa979d75fc166be58 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x3:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b2010 expref: 21 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.002653:0:6490:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e800/0xa979d75fc166be5f lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x3:0x0].0x0 bits 0x10/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b2010 expref: 20 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.002661:0:6490:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x3:0x0].0x0 (ffff961741937200) 00010000:00010000:0.0:1787096582.002664:0:6490:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 10 00010000:00010000:0.0:1787096582.002667:0:6490:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96174340e800, rc 0) 00000100:00100000:0.0:1787096582.002747:0:6490:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850224700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:70e19547-606a-4706-b016-3476e377d7df+20:34328:x1873906345500544:12345-192.168.201.40@tcp:101:lfs.0 Request processed in 472us (595us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096582.002755:0:6490:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 168 00000100:00100000:1.0:1787096582.003856:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:0.0:1787096582.003906:0:6489:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345500672 00000100:00100000:0.0:1787096582.003912:0:6489:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345500672 00000100:00100000:0.0:1787096582.003944:0:6489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 169 00000100:00100000:0.0:1787096582.003950:0:6489:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850225f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:70e19547-606a-4706-b016-3476e377d7df+20:34328:x1873906345500672:12345-192.168.201.40@tcp:101:lfs.0 00010000:00010000:0.0:1787096582.003962:0:6489:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096582.003974:0:6489:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fc00/0xa979d75fc166be66 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe2ded7f867b2017 expref: -99 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.004026:0:6489:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f800/0xa979d75fc166be6d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6489 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.004034:0:6489:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f800/0xa979d75fc166be6d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.004042:0:6489:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f800/0xa979d75fc166be6d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787096582.004070:0:6489:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000400:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787096582.004074:0:6489:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787096582.004096:0:6489:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fc00/0xa979d75fc166be66 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b2017 expref: 22 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.004111:0:6489:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f800/0xa979d75fc166be6d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b2017 expref: 21 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.004119:0:6489:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffff961741937400) 00010000:00010000:0.0:1787096582.004122:0:6489:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787096582.004125:0:6489:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96174340f800, rc 0) 00000100:00100000:0.0:1787096582.004169:0:6489:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850225f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:70e19547-606a-4706-b016-3476e377d7df+21:34328:x1873906345500672:12345-192.168.201.40@tcp:101:lfs.0 Request processed in 219us (315us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096582.004178:0:6489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 169 00000100:00100000:3.0:1787096582.005551:0:3628:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:2.0:1787096582.007965:0:6488:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345500800 00000100:00100000:2.0:1787096582.007979:0:6488:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345500800 00000100:00100000:2.0:1787096582.008045:0:6488:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 170 00000100:00100000:2.0:1787096582.008057:0:6488:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742fd3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:70e19547-606a-4706-b016-3476e377d7df+7:34328:x1873906345500800:12345-192.168.201.40@tcp:101:lfs.0 00010000:00010000:2.0:1787096582.008079:0:6488:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787096582.008099:0:6488:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4a600/0xa979d75fc166be74 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe2ded7f867b201e expref: -99 pid: 6488 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.008167:0:6488:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be7b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.008184:0:6488:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4a800/0xa979d75fc166be27 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0xa979d75fc166be20 expref: 7 pid: 7418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.008209:0:6488:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be7b lrc: 4/1,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.008249:0:6488:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4a800/0xa979d75fc166be27 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0xa979d75fc166be20 expref: 7 pid: 7418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096582.008267:0:6488:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961742cc9880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6488:1873906353971200:0@lo:104: 00000100:00100000:2.0:1787096582.008336:0:6488:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096582.008385:0:6471:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353971200 00000100:00100000:3.0:1787096582.008390:0:6471:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353971200 00000100:00100000:2.0:1787096582.008390:0:6488:0:(client.c:2692:ptlrpc_set_wait()) set ffff961742a350c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787096582.008423:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 43 00000100:00100000:3.0:1787096582.008431:0:6471:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742ccb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6488:x1873906353971200:12345-0@lo:104: 00010000:00010000:3.0:1787096582.008467:0:6471:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ee00/0xa979d75fc166be20 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xa979d75fc166be27 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787096582.008530:0:6471:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787096582.008536:0:6471:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742ccb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6488:x1873906353971200:12345-0@lo:104: Request processed in 107us (201us total) trans 0 rc 0/0 00010000:00010000:0.0:1787096582.008552:0:7858:0:(ldlm_lockd.c:2794:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:3.0:1787096582.008554:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 43 00010000:00010000:0.0:1787096582.008563:0:7858:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ee00/0xa979d75fc166be20 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0xa979d75fc166be27 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096582.008570:0:6488:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961742cc9880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6488:1873906353971200:0@lo:104: 00010000:00010000:0.0:1787096582.008578:0:7858:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96174340ee00 already unused, calling callback (ffffffffc1765ee0) 00010000:00010000:0.0:1787096582.008591:0:7858:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ee00/0xa979d75fc166be20 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0xa979d75fc166be27 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.008596:0:6488:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be7b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787096582.008604:0:7858:0:(mdt_handler.c:3963:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ee00/0xa979d75fc166be20 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0xa979d75fc166be27 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.008636:0:7858:0:(ldlm_request.c:1438:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ee00/0xa979d75fc166be20 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xa979d75fc166be27 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.008643:0:7858:0:(ldlm_request.c:1490:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787096582.008676:0:7858:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ee00/0xa979d75fc166be20 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xa979d75fc166be27 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.008685:0:7858:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffff96174340ee00/0xa979d75fc166be20 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0xa979d75fc166be27 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787096582.008696:0:3635:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff96187c9e7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3635:1873906353971328:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787096582.008733:0:3635:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787096582.008740:0:7858:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096582.008743:0:7858:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787096582.008745:0:7858:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787096582.008819:0:8380:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353971328 00000100:00100000:2.0:1787096582.008823:0:8380:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353971328 00010000:00010000:2.0:1787096582.008841:0:8380:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4a800/0xa979d75fc166be27 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0xa979d75fc166be20 expref: 8 pid: 7418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096582.008864:0:8380:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 63 00000100:00100000:2.0:1787096582.008882:0:8380:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850226680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3635:x1873906353971328:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787096582.008892:0:8380:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787096582.009512:0:8380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4a800/0xa979d75fc166be27 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0xa979d75fc166be20 expref: 8 pid: 7418 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.009525:0:8380:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff961742a27200) 00010000:00010000:2.0:1787096582.009530:0:8380:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be7b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.009549:0:8380:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be7b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.009557:0:8380:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be7b lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.009578:0:8380:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787096582.009602:0:6488:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be7b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.009612:0:6488:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be7b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787096582.009616:0:8380:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850226680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+7:3635:x1873906353971328:12345-0@lo:103:ldlm_bl.0 Request processed in 744us (883us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096582.009629:0:8380:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 63 00000100:00100000:1.0:1787096582.009651:0:3635:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff96187c9e7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3635:1873906353971328:0@lo:103:ldlm_bl.0 00000004:00200000:0.0:1787096582.009677:0:6488:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000401:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787096582.009684:0:6488:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787096582.009727:0:6488:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4a600/0xa979d75fc166be74 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b201e expref: 9 pid: 6488 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.009742:0:6488:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff961845b4be00/0xa979d75fc166be7b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b201e expref: 8 pid: 6488 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.009757:0:6488:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff961742a27200) 00010000:00010000:0.0:1787096582.009759:0:6488:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787096582.009762:0:6488:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff961845b4be00, rc 0) 00000100:00100000:0.0:1787096582.009819:0:6488:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742fd3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:70e19547-606a-4706-b016-3476e377d7df+8:34328:x1873906345500800:12345-192.168.201.40@tcp:101:lfs.0 Request processed in 1765us (4271us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096582.009826:0:6488:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 170 00000100:00100000:1.0:1787096582.011246:0:3627:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:0.0:1787096582.011289:0:6490:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345500928 00000100:00100000:0.0:1787096582.011298:0:6490:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345500928 00000100:00100000:0.0:1787096582.011351:0:6490:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 171 00000100:00100000:0.0:1787096582.011371:0:6490:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961850224e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:70e19547-606a-4706-b016-3476e377d7df+21:34328:x1873906345500928:12345-192.168.201.40@tcp:101:lfs.0 00010000:00010000:0.0:1787096582.011397:0:6490:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096582.011427:0:6490:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e400/0xa979d75fc166be82 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe2ded7f867b2025 expref: -99 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011547:0:6490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fc00/0xa979d75fc166be89 lrc: 3/1,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6490 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011561:0:6490:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fc00/0xa979d75fc166be89 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011572:0:6490:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fc00/0xa979d75fc166be89 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011716:0:6490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e000/0xa979d75fc166be90 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6490 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011741:0:6490:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e000/0xa979d75fc166be90 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011772:0:6490:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e000/0xa979d75fc166be90 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011836:0:6490:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fc00/0xa979d75fc166be89 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011843:0:6490:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fc00/0xa979d75fc166be89 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011849:0:6490:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fc00/0xa979d75fc166be89 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011855:0:6490:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fc00/0xa979d75fc166be89 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011861:0:6490:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96174340fc00 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:0.0:1787096582.011864:0:6490:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fc00/0xa979d75fc166be89 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011887:0:6490:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fc00/0xa979d75fc166be89 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011899:0:6490:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffff961741937400) 00010000:00010000:0.0:1787096582.011902:0:6490:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096582.011906:0:6490:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fc00/0xa979d75fc166be89 lrc: 3/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011912:0:6490:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fc00/0xa979d75fc166be89 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011928:0:6490:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e000/0xa979d75fc166be90 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011940:0:6490:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e000/0xa979d75fc166be90 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011946:0:6490:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e000/0xa979d75fc166be90 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011965:0:6490:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96174340e000 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:0.0:1787096582.011968:0:6490:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e000/0xa979d75fc166be90 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011977:0:6490:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e000/0xa979d75fc166be90 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011985:0:6490:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0xae2548a7 (ffff961741937a00) 00010000:00010000:0.0:1787096582.011988:0:6490:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096582.011991:0:6490:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e000/0xa979d75fc166be90 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.011997:0:6490:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e000/0xa979d75fc166be90 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6490 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.012042:0:6490:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e400/0xa979d75fc166be82 lrc: 1/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b2025 expref: 22 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.012059:0:6490:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffff961741937400) 00010000:00010000:0.0:1787096582.012062:0:6490:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096582.012065:0:6490:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e400/0xa979d75fc166be82 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b2025 expref: 22 pid: 6490 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.012074:0:6490:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96174340e400, rc 0) 00000100:00100000:0.0:1787096582.012158:0:6490:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961850224e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:70e19547-606a-4706-b016-3476e377d7df+21:34328:x1873906345500928:12345-192.168.201.40@tcp:101:lfs.0 Request processed in 797us (916us total) trans 0 rc 301/301 00000100:00100000:0.0:1787096582.012168:0:6490:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 171 00000100:00100000:1.0:1787096582.014301:0:3629:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:0.0:1787096582.014363:0:6489:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345501184 00000100:00100000:0.0:1787096582.014373:0:6489:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345501184 00000100:00100000:0.0:1787096582.014430:0:6489:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 172 00000100:00100000:0.0:1787096582.014465:0:6489:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742d06680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:70e19547-606a-4706-b016-3476e377d7df+21:34328:x1873906345501184:12345-192.168.201.40@tcp:101:lfs.0 00010000:00010000:0.0:1787096582.014489:0:6489:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096582.014491:0:6489:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1787096582.014512:0:6489:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f800/0xa979d75fc166be6d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b2017 expref: 21 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.014528:0:6489:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffff961741937400) 00010000:00010000:0.0:1787096582.014532:0:6489:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787096582.014541:0:6489:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f400/0xa979d75fc166be97 lrc: 2/0,0 mode: --/CW res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe2ded7f867b202c expref: -99 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.014660:0:6489:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e200/0xa979d75fc166be9e lrc: 3/0,1 mode: --/CW res: [0x240000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6489 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.014671:0:6489:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e200/0xa979d75fc166be9e lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.014678:0:6489:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e200/0xa979d75fc166be9e lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.014689:0:6489:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fa00/0xa979d75fc166bea5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6489 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.014694:0:6489:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fa00/0xa979d75fc166bea5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.014709:0:6489:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fa00/0xa979d75fc166bea5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096582.014887:0:6489:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff96187c9e4700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6489:1873906353971456:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787096582.014944:0:6489:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096582.014984:0:7418:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353971456 00000100:00100000:0.0:1787096582.014987:0:6489:0:(client.c:2692:ptlrpc_set_wait()) set ffff961741ec3180 going to sleep for 11 seconds 00000100:00100000:2.0:1787096582.014989:0:7418:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353971456 00000100:00100000:2.0:1787096582.015025:0:7418:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 198 00000100:00100000:2.0:1787096582.015031:0:7418:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff96187c9e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6489:x1873906353971456:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787096582.015099:0:7418:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff96187c9e4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6489:x1873906353971456:12345-0@lo:1000:mdt00_001.0 Request processed in 67us (154us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096582.015107:0:7418:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 198 00000100:00100000:0.0:1787096582.015133:0:6489:0:(client.c:3102:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294967360 gen 2 00000004:00080000:0.0:1787096582.015139:0:6489:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294967359 committed transno 4294967360 00000004:00080000:0.0:1787096582.015168:0:6489:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 4294967360 committed transno 4294967360 00000100:00100000:0.0:1787096582.015193:0:6489:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff96187c9e4700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6489:1873906353971456:0@lo:1000:mdt00_001.0 00010000:00010000:0.0:1787096582.015583:0:6489:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f600/0xa979d75fc166beac lrc: 3/1,0 mode: --/PR res: [0x240000402:0x4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6489 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.015594:0:6489:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f600/0xa979d75fc166beac lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.015602:0:6489:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f600/0xa979d75fc166beac lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787096582.015613:0:6489:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x4:0x0]: returning size 0 00010000:00010000:0.0:1787096582.015640:0:6489:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e200/0xa979d75fc166be9e lrc: 3/0,1 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.015647:0:6489:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e200/0xa979d75fc166be9e lrc: 2/0,0 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.015654:0:6489:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e200/0xa979d75fc166be9e lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.015659:0:6489:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96174340e200 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:0.0:1787096582.015670:0:6489:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e200/0xa979d75fc166be9e lrc: 3/0,0 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.015677:0:6489:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e200/0xa979d75fc166be9e lrc: 4/0,0 mode: CW/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.015687:0:6489:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0x0 (ffff961741937400) 00010000:00010000:0.0:1787096582.015691:0:6489:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787096582.015694:0:6489:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e200/0xa979d75fc166be9e lrc: 2/0,0 mode: --/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.015701:0:6489:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340e200/0xa979d75fc166be9e lrc: 0/0,0 mode: --/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787096582.015713:0:6489:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000255508c7 reply state 00000000d5858992 transno 4294967309 ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fa00/0xa979d75fc166bea5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.015724:0:6489:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff96174340fa00/0xa979d75fc166bea5 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0x5:0x0].0xae2548a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6489 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.015731:0:6489:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x5:0x0].0xae2548a7 (ffff961741937e00) 00010000:00010000:0.0:1787096582.015733:0:6489:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787096582.015736:0:6489:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787096582.015763:0:6489:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f400/0xa979d75fc166be97 lrc: 0/0,0 mode: --/CW res: [0x240000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b202c expref: 23 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.015777:0:6489:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f600/0xa979d75fc166beac lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b202c expref: 22 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.015783:0:6489:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4:0x0].0x0 (ffff961741937f00) 00010000:00010000:0.0:1787096582.015784:0:6489:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787096582.015791:0:6489:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff96174340f600, rc 0) 00000100:00100000:0.0:1787096582.015833:0:6489:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742d06680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:70e19547-606a-4706-b016-3476e377d7df+23:34328:x1873906345501184:12345-192.168.201.40@tcp:101:lfs.0 Request processed in 1395us (1541us total) trans 4294967309 rc 0/0 00000100:00100000:0.0:1787096582.015838:0:6489:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 172 00000100:00100000:3.0:1787096582.023247:0:3628:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:0.0:1787096582.023301:0:6488:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345501440 00000100:00100000:0.0:1787096582.023315:0:6488:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345501440 00000100:00100000:0.0:1787096582.023399:0:6488:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 173 00000100:00100000:0.0:1787096582.023412:0:6488:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742fd2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:70e19547-606a-4706-b016-3476e377d7df+23:34328:x1873906345501440:12345-192.168.201.40@tcp:101:lfs.0 00010000:00010000:0.0:1787096582.023470:0:6488:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787096582.023497:0:6488:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9618821ee200/0xa979d75fc166beba lrc: 2/0,0 mode: --/CR res: [0x240000402:0x4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xe2ded7f867b2033 expref: -99 pid: 6488 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.023619:0:6488:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9618821ef400/0xa979d75fc166bec1 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6488 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.023633:0:6488:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f600/0xa979d75fc166beac lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b202c expref: 24 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.023640:0:6488:0:(ldlm_resource.c:1724:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffff9618821ef400/0xa979d75fc166bec1 lrc: 4/0,1 mode: --/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.023674:0:6488:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f600/0xa979d75fc166beac lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b202c expref: 24 pid: 6489 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.023688:0:6488:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f600/0xa979d75fc166beac lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b202c expref: 24 pid: 6489 timeout: 364 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096582.023703:0:6488:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff96187c9e4700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6488:1873906353971584:192.168.201.40@tcp:104: 00000100:00100000:0.0:1787096582.023837:0:6488:0:(client.c:2692:ptlrpc_set_wait()) set ffff961741ec3480 going to sleep for 11 seconds 00000100:00100000:0.0:1787096582.025095:0:6488:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff96187c9e4700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6488:1873906353971584:192.168.201.40@tcp:104: 00010000:00010000:0.0:1787096582.025121:0:6488:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffff9618821ef400/0xa979d75fc166bec1 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787096582.025278:0:3628:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:0.0:1787096582.025320:0:6473:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345501568 00000100:00100000:0.0:1787096582.025326:0:6473:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345501568 00010000:00010000:0.0:1787096582.025366:0:6473:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f600/0xa979d75fc166beac lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b202c expref: 25 pid: 6489 timeout: 364 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787096582.025404:0:6473:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 64 00000100:00100000:0.0:1787096582.025416:0:6473:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742fd1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:70e19547-606a-4706-b016-3476e377d7df+25:2400:x1873906345501568:12345-192.168.201.40@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787096582.025429:0:6473:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787096582.025436:0:6473:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f600/0xa979d75fc166beac lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b202c expref: 25 pid: 6489 timeout: 364 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.025490:0:6473:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f600/0xa979d75fc166beac lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b202c expref: 25 pid: 6489 timeout: 364 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.025514:0:6473:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff96174340f600/0xa979d75fc166beac lrc: 0/0,0 mode: --/PR res: [0x240000402:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b202c expref: 25 pid: 6489 timeout: 364 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.025534:0:6473:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4:0x0].0x0 (ffff961741937f00) 00010000:00010000:0.0:1787096582.025539:0:6473:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0001_UUID lock: ffff9618821ef400/0xa979d75fc166bec1 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.025549:0:6473:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffff9618821ef400/0xa979d75fc166bec1 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.025585:0:6473:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffff9618821ef400/0xa979d75fc166bec1 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787096582.025609:0:6473:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787096582.025658:0:6473:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742fd1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:70e19547-606a-4706-b016-3476e377d7df+24:2400:x1873906345501568:12345-192.168.201.40@tcp:103:ldlm_bl.0 Request processed in 244us (379us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096582.025666:0:6473:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 64 00010000:00010000:2.0:1787096582.025803:0:6488:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffff9618821ef400/0xa979d75fc166bec1 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.025812:0:6488:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffff9618821ef400/0xa979d75fc166bec1 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787096582.025835:0:6488:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x4:0x0]: returning size 0 00010000:00010000:2.0:1787096582.026006:0:6488:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffff9618821ef400/0xa979d75fc166bec1 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.026015:0:6488:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffff9618821ef400/0xa979d75fc166bec1 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.026035:0:6488:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffff9618821ef400/0xa979d75fc166bec1 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.026042:0:6488:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9618821ef400 already unused, calling callback (ffffffffc17594f0) 00010000:00010000:2.0:1787096582.026046:0:6488:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9618821ef400/0xa979d75fc166bec1 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.026055:0:6488:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffff9618821ef400/0xa979d75fc166bec1 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.026064:0:6488:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4:0x0].0x0 (ffff961741937f00) 00010000:00010000:2.0:1787096582.026067:0:6488:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1787096582.026071:0:6488:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffff9618821ef400/0xa979d75fc166bec1 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.026077:0:6488:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9618821ef400/0xa979d75fc166bec1 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6488 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.026114:0:6488:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffff9618821ee200/0xa979d75fc166beba lrc: 1/0,0 mode: --/CR res: [0x240000402:0x4:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b2033 expref: 23 pid: 6488 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.026124:0:6488:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4:0x0].0x0 (ffff961741937f00) 00010000:00010000:2.0:1787096582.026126:0:6488:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787096582.026139:0:6488:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffff9618821ee200/0xa979d75fc166beba lrc: 0/0,0 mode: --/CR res: [0x240000402:0x4:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.40@tcp remote: 0xe2ded7f867b2033 expref: 23 pid: 6488 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787096582.026159:0:6488:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9618821ee200, rc 0) 00000100:00100000:2.0:1787096582.026198:0:6488:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742fd2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:70e19547-606a-4706-b016-3476e377d7df+22:34328:x1873906345501440:12345-192.168.201.40@tcp:101:lfs.0 Request processed in 2788us (2962us total) trans 0 rc 301/301 00000100:00100000:2.0:1787096582.026214:0:6488:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 173 00000100:00100000:3.0:1787096582.027405:0:3628:0:(events.c:340:request_in_callback()) peer: 12345-192.168.201.40@tcp (source: 12345-192.168.201.40@tcp) 00000100:00100000:0.0:1787096582.027484:0:13534:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906345501824 00000100:00100000:0.0:1787096582.027493:0:13534:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906345501824 00000100:00100000:0.0:1787096582.027533:0:13534:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.40@tcp, seq: 26 00000100:00100000:0.0:1787096582.027539:0:13534:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742fd2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:70e19547-606a-4706-b016-3476e377d7df+22:34328:x1873906345501824:12345-192.168.201.40@tcp:35:lfs.0 00000100:00100000:0.0:1787096582.027767:0:13534:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742fd2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:70e19547-606a-4706-b016-3476e377d7df+23:34328:x1873906345501824:12345-192.168.201.40@tcp:35:lfs.0 Request processed in 227us (366us total) trans 4294967310 rc 0/0 00000100:00100000:0.0:1787096582.027773:0:13534:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.40@tcp, seq: 26 00000100:00100000:3.0:1787096582.222757:0:3637:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787096582.222787:0:3637:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff96187c9e4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3637:1873906353971712:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787096582.222899:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787096582.222983:0:3636:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787096582.222995:0:3636:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961742d05180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3636:1873906353971840:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787096582.223017:0:3634:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff961742cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3634:1873906353971968:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787096582.223034:0:3636:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787096582.223053:0:3634:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787096582.223067:0:6493:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353971840 00000100:00100000:3.0:1787096582.223084:0:6493:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353971840 00000100:00100000:3.0:1787096582.223149:0:6493:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 199 00000100:00100000:3.0:1787096582.223159:0:6493:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742f87480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3636:x1873906353971840:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1787096582.223201:0:6493:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1446375424 avail 1300135936 00000100:00100000:3.0:1787096582.223259:0:6493:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742f87480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3636:x1873906353971840:12345-0@lo:41:osp-pre-1-0.0 Request processed in 101us (224us total) trans 0 rc 0/0 00000100:00100000:3.0:1787096582.223269:0:6493:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 199 00000100:00100000:1.0:1787096582.223322:0:3636:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961742d05180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3636:1873906353971840:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1787096582.223399:0:17665:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353971712 00000100:00100000:0.0:1787096582.223405:0:17665:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353971712 00000100:00080000:2.0:1787096582.223460:0:6484:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787096582.223492:0:8453:0:(service.c:2304:ptlrpc_server_handle_req_in()) unwrap req x1873906353971968 00000100:00100000:1.0:1787096582.223497:0:8453:0:(service.c:2369:ptlrpc_server_handle_req_in()) got req x1873906353971968 00000100:00100000:0.0:1787096582.223520:0:17665:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 138 00000100:00100000:0.0:1787096582.223531:0:17665:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742fd3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3637:x1873906353971712:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1787096582.223557:0:17665:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:2.0:1787096582.223562:0:8367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 139 00000100:00080000:3.0:1787096582.223611:0:6484:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787096582.223666:0:17665:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742fd3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3637:x1873906353971712:12345-0@lo:13:osp-pre-0-1.0 Request processed in 138us (769us total) trans 0 rc 0/0 00000100:00100000:0.0:1787096582.223674:0:17665:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 138 00010000:00080000:1.0:1787096582.223706:0:3637:0:(ldlm_request.c:1663:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff96187c9e4700 x1873906353971712/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 0 dl 1787096598 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-0-1.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787096582.223733:0:3637:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff96187c9e4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3637:1873906353971712:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787096582.223752:0:8367:0:(service.c:2555:ptlrpc_server_handle_request()) Handling RPC req@ffff961742c66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3634:x1873906353971968:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787096582.223764:0:8367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:2.0:1787096582.223793:0:8367:0:(service.c:2613:ptlrpc_server_handle_request()) Handled RPC req@ffff961742c66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3634:x1873906353971968:12345-0@lo:13:osp-pre-0-0.0 Request processed in 224us (742us total) trans 0 rc 0/0 00000100:00100000:2.0:1787096582.223801:0:8367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 139 00010000:00080000:3.0:1787096582.223819:0:3634:0:(ldlm_request.c:1663:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff961742cc9880 x1873906353971968/t0(0) o13->lustre-OST0000-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 0 dl 1787096598 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-0-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787096582.223830:0:3634:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff961742cc9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3634:1873906353971968:0@lo:13:osp-pre-0-0.0 00010000:00010000:1.0:1787096582.414624:0:6476:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1787096582.628815:0:22028:0:(debug.c:651:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-ec test_23b: @@@@@@ FAIL: setstripe failed Debug log: 1348 lines, 1348 kept, 0 dropped, 0 bad.